Optical disk for high resolution and general video recording, optical disk reproduction apparatus, optical disk recording apparatus, and reproduction control information generation apparatus

ABSTRACT

The present invention has an objective of realizing compatibility of an optical disk having a high resolution video signal recorded thereon and a system for reproducing the optical disk, with a conventional system for producing a standard resolution video signal. A high resolution video signal is divided by video division means into a main signal and a sub signal, and the main signal and the sub signal are MPEG-encoded. The stream of the main signal and the stream of the sub signal are divided into 1 GOP or more of frames. First interleave blocks  54  each including 1 GOP or more of the stream of the main signal and second interleave blocks  55  each including 1 GOP or more of the stream of the sub signal are recorded on an optical disk  1 . A high resolution reproduction apparatus reproduces both the first and second interleave blocks to obtain a high resolution video output. A non-high quality picture reproduction apparatus reproduces only the first or second interleave blocks to obtain a standard resolution video output.

TECHNICAL FIELD

The present invention relates to an optical disk having high qualitypicture or standard picture recorded thereon, and a recording andreproduction apparatus for such an optical disk.

BACKGROUND ART

For an optical disk having high quality picture recorded thereon and areproduction apparatus for such an optical disk, systems referred to as480P and 720P for recording progressive data have conventionally beenstudied A conventionally known reproduction control system for anoptical disk uses one MPEG decoder.

First, a first problem of the conventional system will be described.When a conventional optical disk for high quality picture recording isreproduced by a standard reproduction apparatus, normal pictures cannotbe output. The optical disk for high quality picture recording can onlybe reproduced by a high quality picture reproduction apparatus.Accordingly, there is a need to produce two types of optical diskshaving the same contents. In other words, the conventional optical diskfor high quality picture recording is not compatible with a standardvideo reproduction apparatus. Next, objectives of the present inventionwill be described. A first objective of the present invention is forproviding a optical disk for high quality picture recording compatiblewith the standard video reproduction apparatus and a reproduction systemfor such an optical disk.

The compatibility herein can be defined as the relationship between theconventional monaural records and stereo records. That is, a novel 3Doptical disk or high resolution optical disk according to the presentinvention is output with a standard resolution by the existingreproduction apparatus for DVD or the like, and output with a highresolution by a novel reproduction apparatus according to the presentinvention.

Then, a second problem of the conventional system is regarding areproduction control system. By the conventional reproduction controlsystem, one stream is reproduced using one decoder. Accordingly, inorder to connect two streams of a high resolution signal seamlessly,i.e., without stopping the movement of the video, a complicated systemis required. A second objective of the present invention is forproviding a reproduction control for connecting a plurality of streamsseamlessly by a simple procedure.

DISCLOSURE OF INVENTION

An optical disk reproduction apparatus according to the presentinvention is for reproducing a signal recorded on an optical disk. Theoptical disk has, recorded thereon, at least a first video streamrepresenting a low frequency component of the video signal and a secondvideo stream representing at least a high frequency component of thevideo signal, the first video stream includes a plurality of firstinterleave units and the second video stream includes a plurality ofsecond interleave units, each of the plurality of first interleave unitsincludes m1 GOPs (where m1 is an integer of 1 or greater), each of theplurality of second interleave units includes m2 GOPs (where m2 is aninteger of 1 or greater). The optical disk reproduction apparatusincludes a reproduction section for reproducing the first video streamand the second video stream recorded on the optical disk; a divisionsection for dividing the reproduced first video stream into theplurality of first interleave units and for dividing the reproducedsecond video stream into the plurality of second interleave units; adecoding section for decoding the plurality of first interleave units togenerate a first reproduction signal representing the low frequencycomponent of the video signal and for decoding the plurality of secondinterleave units to generate a second reproduction signal representingat least the high frequency component of the video signal; a synthesissection for synthesizing the first reproduction signal and the secondreproduction signal to generate the video signal; and an output sectionfor selectively outputting at least one of the first reproductionsignal, the second reproduction signal, and the video signal. Theabove-described objective is achieved by this.

The plurality of first interleave units may be each corresponded tofirst time information relating to reproduction time, and the pluralityof second interleave units may be each corresponded to second timeinformation relating to reproduction time.

The optical disk reproduction apparatus may further include a referencetime signal generation section for generating a reference time signal; afirst reproduction control section for controlling the reproduction timeof the first reproduction signal in accordance with the differencebetween the reference time signal and the first time information; asecond reproduction control section for controlling the reproductiontime of the second reproduction signal in accordance with the differencebetween the reference time signal and the second time information; andan adjusting section for adjusting the reference time signal so that thereference time signal supplied to the first reproduction control sectionand the reference signal supplied to the second reproduction controlsection represent substantially the same time.

The adjusting section may adjust the reference time signal based onaudio reproduction time information representing the time to reproducean audio signal which is to be output in synchronization with the videosignal.

The adjusting section may adjust the reference time signal based on atleast one of first video reproduction time information representing thetime to reproduce the first reproduction signal and second videoreproduction time information representing the time to reproduce thesecond reproduction signal.

The first reproduction control section may control the reproduction timeof the first reproduction signal by skipping a frame of the firstreproduction signal or by reproducing a frame of the first reproductionsignal in repetition. The second reproduction control section maycontrol the reproduction time of the second reproduction signal byskipping a frame of the second reproduction signal or by reproducing aframe of the second reproduction signal in repetition.

At least one of the first time information and the second timeinformation may include at least one of a PTS, a DTS and an SCR.

The first reproduction signal may correspond to a first pixel number,and the second reproduction signal may correspond to a second pixelnumber, which is larger than the first pixel number. The synthesissection may include a converter for converting the first reproductionsignal into a conversion signal corresponding to the second pixelnumber. The video signal may be obtained by synthesizing the conversionsignal and the second reproduction signal.

The optical disk further may have, recorded thereon, an identifierrepresenting the first pixel number corresponding to the firstreproduction signal, and the converter may convert the firstreproduction signal into the conversion signal in accordance with theidentifier.

The optical disk further may have, recorded thereon, an identifierrepresenting the first pixel number corresponding to the firstreproduction signal. The optical disk reproduction apparatus may furtherinclude a rotation control section for controlling the rotation of theoptical disk. The rotation control section may control the rotation ofthe optical disk in accordance with the identifier.

The optical disk further may have, recorded thereon, an identifierrepresenting that the video signal is obtained by encoding a progressivevideo signal of 24 frames to 30 frames per second. The output sectionmay include a converter for converting at least one of the firstreproduction signal, the second reproduction signal, and the videosignal into a frame signal. The output section may output theprogressive video signal of 60 frames per second by outputting the framesignal in an overlapping manner.

The optical disk reproduction apparatus may further include a buffermemory section for storing the plurality of first interleave units andthe plurality of second interleave units. The buffer memory section mayhave a capacity which is equal to or greater than an amount of data ofthe GOP or GOPs included in the second interleave units.

The buffer memory section may have a capacity which is 1 MB or greater.

An optical disk according to the present invention include, recordedthereon, at least a first video stream representing a low frequencycomponent of the video signal and a second video stream representing atleast a high frequency component of the video signal, wherein: the firstvideo stream includes a plurality of first interleave units, the secondvideo stream includes a plurality of second interleave units, each ofthe plurality of first interleave units includes m1 GOPs (where m1 is aninteger of 1 or greater), and each of the plurality of second interleaveunits includes m2 GOPs (where m2 is an integer of 1 or greater). Theabove-described objective is achieved by this.

The plurality of first interleave units and the plurality of secondinterleave units may be structured so that reproduction time of one ofthe plurality of first interleave units is substantially equal toreproduction time of one of the plurality of second interleave units,the one of the plurality of second interleave units corresponding to theone of the plurality of first interleave units.

An optical disk recording apparatus according to the present inventionincludes a dividing section for dividing a video signal into a firstvideo signal representing a low frequency component of the video signaland a second video signal representing at least a high frequencycomponent of the video signal; an encoding section for generating afirst video stream by encoding the first video signal and for generatinga second video stream by encoding the second video signal, wherein: thefirst video stream includes a plurality of first interleave units, thesecond video stream includes a plurality of second interleave units,each of the plurality of first interleave units includes m1 GOPs (wherem1 is an integer of 1 or greater), and each of the plurality of secondinterleave units includes m2 GOPs (where m2 is an integer of 1 orgreater); a selection output section for selectively outputting theplurality of first interleave units included in the first video streamand the plurality of second interleave units included in the secondvideo stream; and a recording section for recording the signal outputfrom the selection output section on an optical disk. Theabove-described objective is achieved by this.

The division section may include a decoder for decoding the first videostream and a differential calculator for calculating a differentialbetween the video signal and the signal output from the decoder, and mayoutput the signal output from the differential calculator as the secondvideo signal.

The division section may further include a first converter forconverting the video signal into a first conversion signal correspondingto a second pixel number which is smaller than a first pixel numbercorresponding to the video signal, and a second converter for convertingthe signal output from the decoder into a second conversion signalcorresponding to the first pixel number which is larger than the secondpixel number corresponding to the signal output from the decoder. Thedivision section may output the first conversion signal as the firstvideo signal. The differential calculator may calculate the differentialbetween the video signal and the second conversion signal.

The recording section may further record on the optical disk anidentifier representing that the second video signal is output from thedifferential calculator.

The recording section may further record on the optical disk anidentifier representing the first pixel number corresponding to thevideo signal.

The recording section may further record on the optical disk anidentifier representing the second pixel number corresponding to thefirst video signal.

An optical disk recording apparatus according to the present inventionincludes an input section for receiving an encoded first video streamcorresponding to a first pixel number and an encoded second video streamcorresponding to a second pixel number which is different from the firstpixel number, wherein the first video stream includes a plurality offirst interleave units, the second video stream includes a plurality ofsecond interleave units, each of the plurality of first interleave unitsincludes m1 GOPs (where m1 is an integer of 1 or greater), and each ofthe plurality of second interleave units includes m2 GOPs (where m2 isan integer of 1 or greater); a selection output section for selectivelyoutputting the plurality of first interleave units included in the firstvideo stream and the plurality of second interleave units included inthe second video stream; and a recording section for recording thesignal output from the selection output section on an optical disk. Theabove-described objective is achieved by this.

An optical disk reproduction apparatus according to the presentinvention is for reproducing a signal recorded on an optical disk. Theoptical disk has, recorded thereon, at least a first video streamincluding a plurality of first GOPs and a second video stream includinga plurality of second GOPs, each of the plurality of first GOPs includesa plurality of pictures, and each of the plurality of second GOPsincludes a plurality of pictures, The optical disk reproductionapparatus includes a reproduction section for reproducing the firstvideo stream and the second video stream recorded on the optical disk; adecoding section for decoding the first video stream and the secondvideo stream; and an output section for selectively outputting thedecoded first video stream and the decoded second video stream inaccordance with reproduction control information. The reproductioncontrol information indicates that after a first picture included in afinal first GOP among the plurality of first GOPs included in the firstvideo stream is reproduced, a second picture included in a leadingsecond GOP among the plurality of second GOPs included in the secondvideo stream is reproduced, the second picture being different from aleading picture of the leading second GOP. The above-described objectiveis achieved by this.

The decoding section may start decoding the second video stream so thatthe decoding of the second picture has been completed when thereproduction of the first picture is completed.

The reproduction control information may include information ts1representing a position of the first picture, information ts2representing a position of the second picture, and information tsGrepresenting a position of the leading picture of the leading secondGOP. The decoding section may find a decoding start position ta inaccordance with expression ta=ts1−(ts2−tsG), and starts decoding thesecond video stream based on the decoding start position ta.

The reproduction control information may include timing informationrepresenting the timing to start decoding the leading second GOP so thatreproduction completion time of the first picture matches thereproduction start time of the second picture. The decoding section maystart decoding the second video stream based on the timing information.

The decoding section may omit decoding of a picture which is notnecessary for decoding pictures from the leading picture of the leadingsecond GOP to the second picture.

The picture which is not necessary may be a B picture.

The optical disk reproduction apparatus may further include a buffermemory section for storing the first video stream and the second videostream, and the buffer memory section has a capacity which is equal toor greater than an amount of data of 1 GOP.

The optical disk has the reproduction control information recordedthereon. The reproduction section may reproduce the reproduction controlinformation recorded on the optical disk.

The optical disk may further have, recorded thereon, an identifierrepresenting whether or not the reproduction control information isrecorded on the optical disk, and when the identifier represents thatthe reproduction control information is recorded on the optical disk,the reproduction section may reproduce the reproduction controlinformation recorded on the optical disk.

In a fast reproduction mode, when the second picture is not an Ipicture, the output section may prohibit an I picture included in theleading second GOP from being output.

The output section may prohibit a part of the I picture included in theleading second GOP from being output based on I picture reproductionprohibition information.

A reproduction control information generation apparatus according to thepresent invention includes an input section for receiving a first videostream including a plurality of first GOPs and a second video streamincluding a plurality of second GOPs; and a generation section forgenerating reproduction control information which represents that aftera first picture included in a final first GOP among the plurality offirst GOPs included in the first video stream is reproduced, a secondpicture included in a leading second GOP among the plurality of secondGOPs included in the second video stream is reproduced, the secondpicture being different from a leading picture of the leading secondGOP. The above-described objective is achieved by this.

The reproduction control information may include informationrepresenting the number of pictures from the leading picture of theleading second GOP to the second picture.

The reproduction control information may include informationrepresenting the time to reproduce the leading picture of the leadingsecond GOP and the time to reproduce the second picture of the leadingsecond GOP.

The reproduction control information may include timing informationrepresenting the timing to start decoding the leading second GOP so thatreproduction completion time of the first picture matches thereproduction start time of the second picture.

The timing information may represent the timing to start decoding theleading second GOP when a picture which is not necessary for decodingpictures from the leading picture of the leading second GOP to thesecond picture is not decoded.

The picture which is not necessary may be a B picture.

An optical disk recording apparatus according to the present inventionincludes a generation section for generating reproduction controlinformation; and a recording section for recording the reproductioncontrol information on an optical disk having, recorded thereon, a firstvideo stream including a plurality of first GOPs and a second videostream including a plurality of second GOPs. The reproduction controlinformation represents that after a first picture included in a finalfirst GOP among the plurality of first GOPs included in the first videostream is reproduced, a second picture included in a leading second GOPamong the plurality of second GOPs included in the second video streamis reproduced, the second picture being different from a leading pictureof the leading second GOP. The above-described objective is achieved bythis.

An optical disk recording apparatus according to the present inventionincludes an editing section for editing a first video stream including aplurality of first GOPs and a second video stream including a pluralityof second GOPs so that at least one picture unnecessary for reproductionis deleted in accordance with the reproduction control information; anda recording section for recording the edited first video stream and theedited second video stream on an optical disk. The reproduction controlinformation represents that after a first picture included in a finalfirst GOP among the plurality of first GOPs included in the first videostream is reproduced, a second picture included in a leading second GOPamong the plurality of second GOPs included in the second video streamis reproduced, the second picture being different from a leading pictureof the leading second GOP. The above-described objective is achieved bythis.

The at least one picture unnecessary for reproduction may include apicture, of the first video stream, after the first picture, and apicture, of the second video stream, before the second picture.

The at least one picture unnecessary for reproduction may furtherinclude a picture which is not necessary for decoding pictures from theleading picture of the leading second GOP in the second video streamuntil the second picture.

The at least one picture unnecessary for reproduction may be a Bpicture.

The recording section may record the edited first video stream and theedited second video stream in continuous regions of the optical disk.

The recording section may record the reproduction control information onthe optical disk.

The recording section may record the reproduction control information ona medium other than the optical disk.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of a 720P/480P hierarchical recordingapparatus in one example according to the present invention.

FIG. 2 is a block diagram of a 480i/480P/720P (60) reproductionapparatus in one example according to the present invention.

FIG. 3 is a block diagram of a 480P/720P (24/60) reproduction apparatusin one example according to the present invention.

FIG. 4 is a block diagram of a reproduction apparatus (720P output) ofhorizontal direction synthesis system in one example according to thepresent invention.

FIG. 5 is a block diagram of a three hierarchical layer optical diskrecording apparatus in one example according to the present invention.

FIG. 6 is a block diagram of a recording and reproduction apparatus of aframe-based reproduction control system in one example according to thepresent invention.

FIG. 7 shows a process for recording steams by a recording apparatus ofa reproduction control information recording system in one exampleaccording to the present invention.

FIG. 8 is a diagram comparing reproduction of an optical disk in oneexample according to the present invention by an existing reproductionapparatus and by a reproduction apparatus according to the presentinvention.

FIG. 9 is a graph illustrating the relationship between the recordingtime period and the capacity of an optical disk in one example accordingto the present invention.

FIG. 10 is a block diagram of a hierarchical reproduction apparatus of a480P reproduction mode in one example according to the presentinvention.

FIG. 11 is a table showing a data structure of reproduction controlinformation in one example according to the present invention.

FIG. 12 is a diagram illustrating a process for recording a plurality ofstreams by a recording apparatus in one example according to the presentinvention and a process for reproducing the plurality of streams by areproduction apparatus in one example according to the presentinvention.

FIG. 13 is a flowchart illustrating reproduction control of two streamsperformed based on the reproduction control information by areproduction apparatus in one example according to the presentinvention.

FIG. 14 is a table showing a data structure of the reproduction controlinformation when the time stamps of the streams are continuous in oneexample according to the present invention.

FIG. 15 is a diagram illustrating a process for recording andreproduction performed by a recording and reproduction apparatus in oneexample according to the present invention.

FIG. 16 is a flowchart illustrating a process for editing and generationof reproduction control information performed by a recording apparatusin one example according to the present invention.

FIG. 17 is a table showing a data structure of a picture identifier,representing information including resolution, of management informationdata in one example according to the present invention.

FIG. 18 is a block diagram showing an MPEG decoder of a reproductionapparatus of a different system in one example according to the presentinvention;

FIG. 19 is a diagram illustrating the principle of multiple anglepicture data division multiplex recording system in one exampleaccording to the present invention.

FIG. 20 is a diagram illustrating a method for recording horizontal andvertical interpolation information in interleave blocks after beingdivided in one example according to the present invention.

FIG. 21 is a view illustrating the principle of an MADM system fordividing a signal into two in a horizontal direction in one exampleaccording to the present invention.

FIG. 22 is a diagram illustrating picture synthesis control performed bya reproduction apparatus in one example according to the presentinvention.

FIG. 23 is a diagram showing signal arrangement for outputting aprogressive signal, an NTSC signal and a HDTV signal in one exampleaccording to the present invention.

FIG. 24 is a timing diagram of reproduction of progressive, 3D and widesignals with respect to the data amount in buffer in one exampleaccording to the present invention.

FIG. 25 is a block diagram of a reproduction apparatus in one exampleaccording to the present invention in an interlace video signal outputmode.

FIG. 26 is a flowchart illustrating a method for performing AVsynchronization of a first decoder and a second decoder in one exampleaccording to the present invention.

FIG. 27 is a flowchart illustrating a method for controlling two buffersections in one example according to the present invention.

FIG. 28 is a timing diagram showing a data stream which is reproducedand output after processed with buffering and decoding by the decoder inone example according to the present invention.

FIG. 29 is a flowchart showing a detailed process for reproducing aprogram chain group by a system control section M1-9 in one exampleaccording to the present invention.

FIG. 30 is a block diagram showing a structure of a part of an AVsynchronization control 12-10, the part performing AV synchronization,in one example according to the present invention.

FIG. 31 is a block diagram of a data decoding section in one exampleaccording to the present invention.

FIG. 32 shows a signal format of a picture identifier in one exampleaccording to the present invention.

FIG. 33 is a flowchart illustrating a process for STC switching forseamless connection in one example according to the present invention.

FIG. 34 is a view illustrating processing of a horizontal filter circuitin one example according to the present invention.

FIG. 35 is a block diagram showing a structure of an optical diskreproduction apparatus in one example according to the presentinvention.

FIG. 36 is a structural view of a video decoder in one example accordingto the present invention.

FIG. 37 shows a data structure of an optical disk in one exampleaccording to the present invention.

FIG. 38 is a timing diagram of video reproduction in one exampleaccording to the present invention.

FIG. 39 is a block diagram showing a structure of an optical diskreproduction apparatus in one example according to the presentinvention.

FIG. 40 is a structural view of an audio decoder in one exampleaccording to the present invention.

FIG. 41 shows a data structure of an optical disk in one exampleaccording to the present invention.

FIG. 42 is a timing diagram of audio and video reproduction in oneexample according to the present invention.

FIG. 43 shows an optical disk reproduction apparatus in one exampleaccording to the present invention.

FIG. 44 is a structural view of a video decoder in one example accordingto the present invention.

FIG. 45 is a timing diagram of video reproduction in one exampleaccording to the present invention.

FIG. 46 is a block diagram showing a structure of an optical diskreproduction apparatus in one example according to the presentinvention.

FIG. 47 is a structural view of a video decoder in one example accordingto the present invention.

FIG. 48 is a structural view of a video decoder in one example accordingto the present invention.

FIG. 49 is a structural view of a video decoder in one example accordingto the present invention.

FIG. 50 is a block diagram showing a structure of an optical diskreproduction apparatus in one example according to the presentinvention.

FIG. 51 is a structural view of an audio decoder in one exampleaccording to the present invention.

FIG. 52 shows a data structure of an optical disk in one exampleaccording to the present invention.

FIG. 53 is a timing diagram of audio and video reproduction in oneexample according to the present invention.

FIG. 54 is a timing diagram of operation frequencies of audioreproduction in one example according to the present invention.

FIG. 55 is a timing diagram of operation frequencies of audioreproduction in one example according to the present invention.

FIG. 56 is a diagram illustrating a flow of a stream in a reproductionapparatus in one example according to the present invention.

FIG. 57 is a flowchart illustrating a process for MPEG encoding,editing/reproduction control information generation, and reproductioncontrol performed by a recording and reproduction apparatus in oneexample according to the present invention.

FIG. 58 is a block diagram of a recording and reproduction apparatus ofa frame-based reproduction control system in one example according tothe present invention.

FIG. 59 is a diagram illustrating a process for deleting an unnecessaryframe in one example according to the present invention.

FIG. 60 is a block diagram of a reproduction apparatus of a mutualauthentication system and a TV monitor in one example according to thepresent invention.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, the present invention will be described by way of exampleswith reference to drawings.

EXAMPLE 1

(720P/480P Hierarchical Recording and Reproduction System)

With reference to FIG. 1, a specific hierarchical recording apparatusfor handling two hierarchical layers of 720P and 480P will be described.Later, a method for recording a HDTV signal in a hierarchical manner inthe state where the HDTV signal is divided into a plurality of signalswill be described, with reference to FIG. 20.

In the case of a movie signal, specifically, an original 720P videosignal of 60 frames per second, the signal is input and then has extraframes deleted by a 3-2 pull-down section 746. As a result, a 720P (24P)signal 703 of 24 frames per second is obtained. In the case of a normal60P video signal, the 3-2 pull-down section is bypassed. Herein, 60Prefers to 60 frames per second. The 720P video signal 703 having1280×720 pixels is processed by a 720P/480P down-converter 704 asfollows. First, the number of vertical lines is reduced to 720×⅔=480 bya vertical filter 705. Then, the number of pixels is reduced to 1280 ×9/16=720 pixels by a horizontal filter 706. Thus, the 720P video signal703 is converted into a 480P video signal 707 having 720×480 pixels.Such a low resolution 480P video signal is encoded by an MPEG encoder708 for 480P into a compression MPEG signal. Then, the compression MPEGsignal is decoded back into a 480P video signal 710 by an MPEG decoder709. This signal is enlarged to 3/2 times and 16/9 times respectively bya vertical filter 712 and a horizontal filter 713 in a 480P/720Pup-converter 711, and thus is converted into a 720P high resolutionvideo signal 714. The original 720P video signal 703 and the 720P videosignal 714 obtained by MPEG encoding and decoding aredifferential-calculated by a calculation circuit 715 in a differentialsignal processing 720, and thus differential information 716 isobtained.

The differential information 716 is encoded by a second MPEG encoder 717for 720P into a GOP-based video signal including an intraframe (ipicture) and a differential frame (P or B). This signal is divided bymultiplex means 719 into GOP-based second interleave blocks 718 a and718 b including 1GOP to nGOP. An MPEG stream of a basic signal encodedby the first MPEG encoder 708 for 480P in a basic signal processingsection 721 is made into a 480P GOP-based MPEG stream and then dividedby the multiplex means 719 into first interleave blocks 722 a and 722 b.The first interleave blocks 722 a and 722 b are interleaved into thesecond interleave blocks 718 a and 718 b; i.e., the first interleaveblocks 722 a and 722 b and the second interleave blocks 718 a and 718 bare alternately arranged. The resultant signal is recorded on a disk 724such as a DVD or the like by recording means 723. Also recorded at thispoint are a hierarchical recording identifier 725 indicating the startpoint and the termination point and specified interleave blockreproduction prohibition information 726 for prohibiting the secondinterleave blocks 718 a and 718 b including the differential informationfrom being reproduced by the conventional reproduction apparatus. Theidentifier and information are recorded in overall managementinformation 224 and each of VOBs as shown in FIG. 23.

When the disk 724 is reproduced by the existing reproduction apparatusbased on the DVD standards as shown in FIG. 8, the interleave blocks 722a and 722 b are regarded as a first angle and reproduced. Thereproduction signal is decoded by MPEG data 727, and thus NTSC or 480P(24 frames) video signal is reproduced. As shown in FIG. 23, thespecified interleave block reproduction prohibition information 726 forprohibiting specified interleave blocks including the differentialinformation from being reproduced, for example, an angle switchingprohibition flag is recorded. Accordingly, even if the userinadvertently operates the reproduction apparatus, reproduction of asecond angle, i.e., the second interleave unit is prohibited. In otherwords, the differential information for 720P is automatically preventedfrom being reproduced by the existing DVD reproduction apparatus. Whenthe differential information for 720P is reproduced in error,malfunction occurs since this signal cannot be normally reproduced by afirst MPEG decoder for 480i of the existing reproduction apparatus. Thistype of trouble is avoided by the present invention. In this case,information on connection to the second interleave blocks may beintentionally excluded from the management information 224, which isreferred to as the navigation information in the DVD standards.

The above-described effect is also useful when a 720P signal itself isrecorded in the second interleave blocks. In this case, the 720P signalis directly input to the MPEG encoder 717 as shown by the arrowindicated with “*” in FIG. 1.

In this manner, when the disk 724 is reproduced by the existing DVDreproduction apparatus, a video signal is reproduced at the qualityequivalent to NTSC, which is obtained by reproducing an existing DVDdisk; and furthermore erroneous reproduction of information which cannotbe normally reproduced by the existing. DVD reproduction apparatus, suchas a differential signal or a 720P signal, is prevented. Thus,bidirectional compatibility is realized.

A 480P signal itself may be recorded in the second interleave blocksinstead of the 720P signal. In this case, the first interleave blocksare reproduced and thus a 480i (NTSC) signal is output by theconventional reproduction apparatus. By a reproduction apparatusaccording to the present invention, a 480i signal from the firstinterleave blocks or a 480P signal from the second interleave blocks isreproduced, or both of them can be reproduced.

When a reproduction apparatus according to the present invention isused, a basic signal is reproduced from the first interleave blocks 722a and 722 b, which is referred to as the first angle in the DVDstandards. A differential signal and a 720P signal are reproduced fromthe second interleave blocks 718 a and 718 b, which is referred to asthe second angle in the DVD standards. From the first angle, a 480Pvideo signal 729 is output by an MPEG decoder 728 for 480P: and from thesecond angle, a 720P video signal 731 or a 720P signal as a differentialsignal is reproduced by an MPEG decoder 730 for 720P. These two videosignals having a different number of pixels are synthesized by asynthesis section 732 or output as they are, and thus decoded into anoriginal 720P video signal 733 to be output.

In this manner, when the hierarchical recording disk 724 is reproducedby the reproduction apparatus according to the present invention, a 720Pvideo signal is output. Thus, a HDTV signal such as a 720P signal can berecorded while compatibility with the conventional reproductionapparatus is maintained.

When a 480P signal itself is recorded in the second interleave blocks, a480P signal having a density twice as high as that of an NTSC signal isreproduced.

With reference to FIG. 3, a more specific operation of the reproductiondescribed with reference to FIG. 8 will be described. The blocks whichhave already been described will not be described again.

The disk 724 has a basic signal and a differential signal recordedthereon alternately after being divided on an nGOP-by-nGOP basis by themultiplex means 719 shown in FIG. 1 and interleaved. This signal isdivided into the first interleave block 722 a and the second interleaveblock 718 a, i.e., the basic signal and the differential signal, by adivision section 734 of the reproduction apparatus shown in FIG. 3.Then, the basic signal and the differential signal are stored in a firstbuffer memory 735 and a second buffer memory 736 respectively. Then,respective time information is extracted from a time informationextraction section 793. A VTS synchronization section 780 sets firstbasic time information and second basic time information in the firstdecoder 728 and the second decoder 730 so that the two signals are insynchronization with each other. Thus, output signals from the twodecoders are synchronized. In this case, when the hierarchical recordingidentifier 725 is detected, an identification information processingsection 745 recognizes that a first reproduction signal which is thedecoded signal of the first stream is a basic signal having a smallernumber of pixels and that a second reproduction signal which is thedecoded signal of the second stream is a differential signal having alarger number of pixels and having the differential information from thebasic signal. Thus, the identification information processing section745 give the synthesis section 732 an instruction regarding anup-converter 738 and an instruction of addition.

The MPEG decoder 728 for 480P and the MPEG decoder 730 respectivelydecode the signals into a 480P (24) signal and a 720 (24 frames) signal.The decoded signals have 24 frames/sec. or 30 frames/sec. The signalsare processed by 2-3 conversion sections 737 a and 737 b so as to outputthe same frame twice, and thus a 480P signal 729 of 60 frames/sec. and a720P signal 731 having the differential information are obtained. The480P signal 729 is up-converted into a 720P signal 739 by the 480P/720Pup-converter 738 and added to the 720P signal 731 having thedifferential information by an addition section 740, and thus theoriginal 720P video signal 733 is obtained. The addition section 740calculates, for example, as shown in the figure. Where the pixels of therespective signals are a and b, (a+b)/2 is performed to obtain theoriginal 720P video signal 733. The calculation performed by thesynthesis section 732 may be different from (a+b)/2.

In this case, the MPEG decoding signals may be kept to have 24frames/sec. without being converted by the 2-3 conversion sections 737 aand 737 b to have 60 frames/sec. and after synthesis, converted to have60 frames/sec. by a 2-3 conversion section 741. In such a case, theamount of data of the video signal is advantageously reduced to half,and the processing ability of the digital processing circuit can bereduced to half.

With reference to FIGS. 1 and 3, the method for recording andreproducing a 720P signal of 24 frames/sec. such as a movie signal in ahierarchical manner has been described. This method has significantadvantages. The HDTV format includes the 1080i system and the 720Psystem. As shown in FIG. 9, a 1080i signal (24 frames) used for moviescan be recorded only for 90 minutes since a two-layer DVD has a capacityof 8.5 GB as indicated by curve 742 a.

By contrast, a 720P signal (24 frames) can be recorded for 150 minutesas indicated by curve 742 b. A 480P signal (60 frames) can be recordedfor 150 minutes as indicated by curve 742 c. Disks for movies areconsidered to be meaningless unless each has a recording capacity of 120minutes or more. The 720P (24)/480P hierarchical recording disk has aneffect that an HDTV movie title can be accommodated in one DVD.

In the example shown in FIG. 3, a 480P signal having the basicinformation of a 720P signal is recorded in the first interleave blocks,and the differential information between the 720P and 480P signals isrecorded in the second interleave blocks. For a disk having a 720Psignal as it is in the second interleave blocks is reproduced, theoutput from the second decoder 730 can be output as it is as shown byarrow indicated with “*” in FIG. 3. The determination to do this is madeby the identification information processing section 743 based on anidentifier. In this case also, an effect equivalent to completecompatibility is obtained. This system has a lower recording efficiencybut has the effect of simplifying the processing circuit for recordingand reproduction and the effect of the complete compatibility.

With reference to FIG. 60, an example in which a decoder is mounted on aTV monitor 798 will be described. The basic operation is the same asthat described with reference to FIG. 3, and only the different partswill be described. In a reproduction apparatus 743 a, a signal beforedecoding is encrypted by an encryption encoder 795 using an encryptionkey 799 a and sent from a communication interface section 796 a througha network 798 to a communication interface section 796 b in the TVmonitor 798. Before this operation, mutual authentication sections 794 aand 794 b communicate to each other to authenticate each other. Thisoperation may be referred to as a handshake. In the case where mutualauthentication is confirmed and thus both sides determine that the otherside is proper, the mutual authentication sections 794 a and 794 bprovide the encryption encoder 795 and an encryption decoder 797 withthe encryption keys 799 a and 799 b and also permit the communicationinterface 796 a and 796 b to communicate. Thus, encryption data is sentand received, and the keys of the encryption data are unlocked.Therefore, the first stream and the second stream are respectively sentto the first decoder 728 and the second decoder 730. The determinationto conduct this processing is made by the identification informationprocessing section 745 based on an identifier 744 which is separatelysent. When the first stream is of a 480P signal and the second stream isof a 720P differential signal as described above, up-conversion andsynthesis calculation are performed, and a 720P signal is output to a TVmonitor 798 a. When an identifier indicating that the second stream isof a 480P differential signal is received, the two streams aresynthesized to output a 480P signal. When an identifier indicating thatthe streams are of a 3D signal, a 3D signal is output and displayed onthe TV monitor 798 a. The 3D signal is obtained by time-based synthesis,where the first stream is set for the left eye and the second stream isset for the right eye.

According to this system, even when two streams are encrypted, thestreams are processed, for example, synthesized by the TV monitor usingthe identifier 744. Thus, the original picture can be obtained withoutviolating the copyright protection secured by the encryption.

Next, with reference to FIG. 10, an operation for reproducing a disk 724a having a 480P (60 frames/sec.) recorded thereon by the reproductionapparatus according to the present invention will be described. Commonparts as those in FIG. 3 will not be described.

(Sum and difference system—FIG. 19)

With reference to FIG. 19, the concept of the sum and difference systemwill be described. This system is referred to as multiple angle videodata division multiplex system (MADM) since a video signal is dividedinto vertical or horizontal high frequency and low frequency componentsand recorded in each of multiple angles. As shown in FIG. 19, a signalis divided into a basic signal (sum signal) and a sub signal(differential signal) by a sum calculation section 141 and adifferential calculation section 143. The resultant signals areMPEG-encoded and then alternately recorded as interleave blocks in unitsof 1 GOP. At this point, the amount of the information can be reduced by20% by performing a 3-2 conversion of the basic signal and the subsignal in synchronization with each other. It is efficient to use, asthe basic signal. “IBBPBBPBBPBBPBB” which is shown as a main GOPstructure 244 used for the ordinary MPEG encoding. In this structure, anI frame 246, B frames 248 and P frames 247 are alternately arranged. Inthe case of the differential signal, however, experiments have shownthat it is efficient to have a structure including only I frames 246 andP frames 247 due to the profile pattern, for example, “IPPPPPPPIPPPPPPP”shown as a sub GOP structure 245. The efficiency is improved by changingthe setting for the sub GOP structure.

FIG. 19 shows an example in which a 480P video signal is divided intotwo in a vertical direction. FIG. 21 (described below) shows an examplein which a 480P video signal is divided into two in a horizontaldirection. In an alternative manner, a 60-frame 480P signal may bedivided by frame division means into 30 odd frames and 30 even frames.In this case, the respective 30P signals are converted into two 60-fieldinterlace signals, and each of the signals are MPEG-encoded and recordedin the MADM system. Such encoding is performed in a progressive manner,and therefore encoding efficiency is improved as in the case of themovie. Thus, the recordable time period of the same disk is extended.

When such a signal is reproduced by a non-MADM reproduction apparatus, a30P (one-channel) 525 interlace signal is reproduced in a first channel.Such a signal lacks necessary frames and is distorted.

When such a signal is reproduced by an MADM reproduction apparatus, a30P signal is reproduced as a basic signal and another 30P signal isreproduced as a sub signal. These two 30-frame signals are synthesizedinto a 60-frame normal 480P signal by frame synthesis means including aframe buffer, and then output.

When a line doubler is added to an output section for the 480P signal, a1050P video signal is obtained.

When a 525 interlace signal is input to a sum signal section of thesynthesis section of the MADM reproduction apparatus and the value of 0is input to a differential signal section of the synthesis section, a480P video signal is obtained. Such a manner of input has the sameeffect as the line doubler. This method allows even a 525 interlacesignal to be output as a 480P signal. Accordingly, all types of picturescan be viewed by simply connecting one cable to a progressive inputterminal of the MADM reproduction apparatus.

In FIG. 19, ½(A+B) and ½(A−B) are used as expressions for calculationfor a two-tap filter. The division frequency corresponds to about 300scanning lines.

As shown in FIG. 19, the disk 724 a having a 480P signal in the state ofbeing divided into two signals by sum and differential calculations andrecorded in two block groups, i.e., the first interleave blocks and thesecond interleave blocks is reproduced. The signal is divided into a480i signal as a basic signal and a 480i signal as a differential signalby the division section 734. The signals are respectively decoded by theMPEG decoders 728 and the MPEG decoder 730 to obtain a 480i signal 729 aand a 480i differential signal 731 a. The addition section 740 performsthe calculation of (a+b)/2 to synthesize the two 480i signals. Thus, a480P (60 frames) synthesis signal 733 a is output.

The disk 724 includes 480i/480P/720P identification information 744(FIG. 17) recorded in a toc section or the like thereof. The480i/480P/720P identification information 744 includes three signals inthe case of the 480i signal, the 480P signal and the 720P signal, andalso shows which resolution of differential signals are recorded. Theidentification information processing section 743 processes thisinformation to determine in which sector address of the disk main data(main signal) and sub data (differential signal) of the hierarchicaldata are recorded. The identification information processing section 743then sends information on the start point and the like to the synthesissection 732. The synthesis section 732 performs a synthesis calculationof the main data and the sub data from the start point of the 480Psignal and outputs a 480P (60 frames/sec.) signal.

As shown in FIG. 17, row of Vts=6, it is recorded on the disk that atthe start point of the 720P signal, 720P-main is the first interleaveblock, and 720P-sub is the second interleave block. The identificationinformation processing section 743 identifies this information. Thecalculation section 740 performs a synthesis calculation of the 720Psignal, for example, (a+b)/2 from the start time stamp of the 720Psignal, using the time stamps of the main signal and the differentialsignal from the MPEG decoders 728 and 730, and then outputs a 720Psignal.

When a 480P identifier is recorded as the identification information 744(FIG. 17), the identification information processing section 745 sends a480i signal decoding instruction to the MPEG decoder 730 and causes theMPEG decoder 730 to decode the 480i signal. Then, a differential signal731 a of the 480i signal is decoded and synthesized by the synthesissection 732. Thus, a 480P (60 frames/sec.) is output.

In this manner, the MPEG decoder 730 performs the 480i processing(480P-30 frames/sec.) or the 720P processing in accordance with theidentification information. Thus, the main signal and the differentialsignal of the 480i signal, and the main signal and the sub signal of the720P signal, can be decoded by two MPEG decoders in total. This has theeffect of simplifying the structure of the apparatus.

The 480P reproduction mode shown in FIG. 10 does not use the 480P/720Pup-converter 738 in the synthesis section 732, but allows the decoded480P (60) signal to be up-converted to a 720P signal by the 480P to 720Pup-converter 738 and thus displayed on a HD video projector for 720P orthe like. Thus, the scanning lines are advantageously more unlikely tobe viewed. In this case, one 480P to 720P up-converter 738 can be usedfor 720P signal synthesis and 480P to 720P up-conversion. Thus, the 480Psignal can advantageously be up-converted into a 720P signal withoutadding any element.

(720P/480P/480i three hierarchical layer recording apparatus)

With reference to FIG. 5, a structure and an operation of the 720P (60frames/sec.)-type three hierarchical layer recording apparatus will bedescribed. The structure and the operation are substantially the same asthose in FIG. 1, and only the different parts will be described. First,an input signal is a 720P signal of 60 frames/sec. Accordingly, afterbeing 480P down-converted, the input signal is a 480P signal of 60frames/sec. This signal is input to a basic signal processing section721 a and processed by a division section 747. Where the pixel data ofthe n'th line is a and the pixel data of the (n+1)th line is b, thedivision section 747 uses the calculation result of (a+b)/2 for the m'thline of a 480i video signal 748 a and the calculation result of (a−b)/2for the m'th line of a 480i video signal 748 b, and thus obtains a mainsignal and a sub signal of an NTSC signal. These signals arerespectively encoded by the MPEG encoders 708 a and 708 b, and decodedby the MPEG decoders 709 a and 709 b into decoded signals 749 a and 749b. The signals are synthesized by a synthesis section 748 into a 480Psignal 710. The 480P signal is up-converted into a 720P signal 714, anddifferential information is obtained. The differential information isMPEG-encoded to obtain data as third interleave block data 718 a and 718b. This procedure is substantially the same as that in FIG. 1 exceptthat the frame rate is 60 frames/sec. instead of 24 frames/sec.

The 480i MPEG stream is divided by multiplex means 719 a into interleaveblocks on an nGOP basis. The nGOPs are interleaved in the order from a480i signal (first interleave blocks 722 a of a basic signal), then a480i signal (second interleave blocks 750 a of a differential signal),and then a 720P (third interleave blocks 718 a of a differentialsignal), and recorded on the disk 724 such as a DVD.

In this case, the multiplexed signals are modulated by an 8VSB, QAM orOFDM modulation section 751 and transmitted from a transmission section752. Thus, hierarchical broadcasting can be performed. The signals maybe multiplexed by time division based on a time domain defined by thebroadcasting instead of based on a GOP.

In this manner, a 480i/480P (60)/720P three hierarchical layer disk orhierarchical broadcasting is realized.

With reference to FIG. 2, an operation for reproducing the disk 724 awill be described. Since identical elements with those in FIG. 3 areincluded, the identical elements will not be described. A signalreproduced from the disk 724 a or received by a receiving section 753and demodulated by a demodulation section 754 is divided by the divisionsection 734 into three streams based on the above-mentioned interleaveblocks. The three streams are decoded by the three MPEG decoders 728 a,728 b and 730 through buffers 735 a, 735 b and 736. Then, three signals,i.e., the 480i signal 749 a, the 480i differential signal 749 b, and the720P differential signal 731 are obtained as a result of demodulation.By subjecting the 480i basic signal 749 a and the 480i differentialsignal 749 b to the calculations of (a+b) and (a−b) by a synthesissection 732, a 480P (60 frames/sec.) video signal 729 can be obtained.This signal and the above-mentioned 720P differential signal 731 aresynthesized into a 720P output 733 a. The procedure is described aboveand will not be repeated.

In this manner, three types of outputs of 480i output 749 a, 480P output729 and 720P output 733 a having different resolutions can be obtainedfrom the disk 724 a. The user can select the output by the grade of themonitor reproduction apparatus. That is, the 480i (NTSC) grade output isobtained by the existing reproduction apparatus, the 480P (60frames/sec.) is obtained by the reproduction apparatus for 480Paccording to the present invention, and the 720P (60 frames/sec.) isobtained by the reproduction apparatus for 720P according to the presentinvention. Thus, the complete compatibility is realized.

In FIG. 2, when the identification information processing section 745detects a high resolution identifier, the rotation rate of the motor israised through a system control section 21 and a rotation controlcircuit 35. A high resolution signal can be reproduced by raising therotation rate in accordance with the identifier. The rotation rate israised to 1× for reproduction of a standard picture, 2× for 480P and720P (24P), and 3× to 4× for 720P (60P). The effect of power saving isprovided. When an NTSC grade signal is reproduced, the system controlsection 21 stops or operates, at a low rate, clocks of the 720P MPEGdecoder 730, the 480i MPEG decoder 728 b, and the synthesis section 732,which are not necessary. Thus, power consumption can be significantlyreduced. When an ATPS 84 of the audio time stamp of audio data isreceived by an AV synchronization control section 158 and a videopresentation time stamp VPTS for each of the MPEG decoders is createdbased on the time information and set in registers 39 a, 39 b and 39 aof the decoders, frames can be synchronized for reproduction from thedecoders. In order to synchronize the vertical blanking, a decodersynchronization section 794 simultaneously resets the horizontal andvertical synchronization of the decoders. The pictures from the decoderscan be synchronized on a dot-by-dot basis. Specific synchronizationmethods of audio and video signals will be described later.

From the disk 724 a, a first resolution identifier indicating a lowresolution of NTSC signals or the like of the picture of the firststream and a second resolution identifier indicating a high resolutionof 720P signals or the like of the second and third streams arereproduced. The system control section 21 determines by calculationwhich processing is to be performed by the up-converter 738 in thesynthesis section 732 among 480P to 720P, 480P to 1080i, 480P to 1080P,and 720P to 1080P, and indicates the result to the synthesis section732. In actuality, various first resolution identifiers exist such as,for example, 704×480 and 720×480. This has an effect that theup-converter operates at the optimum ratio. Needless to say, a simplesystem structure in which an identifier indicating the ratio of theup-converter is recorded and reproduced can be adopted.

The reproduction apparatus 743 a in FIG. 2 can output three resolutionsof signals (i.e., 480i (NTSC) signal of the first stream, 480P (60P)signal 729 of the first and second streams, and 720P (60P) signal 733 aof the first, second and third streams) simultaneously or at differenttiming. Such a reproduction apparatus can be used with monitors havingvarious resolutions.

Especially, since a 480P signal 729 can be converted into a 720P signalby the up-converter 738 of the synthesis section 732, the 720P signalobtained as a result of conversion of the 480P signal can be obtainedwithout adding any circuit.

In the case where a receiving section 753 and a demodulation section 754are added to the hierarchical reproduction apparatus, a receivingapparatus for receiving a hierarchical signal such as a TV signal,demodulating the signal and outputting three resolutions of videosignals can be provided.

(Wide 480P)

With reference to FIG. 21, a concept of the MADM system in which asignal is divided in a horizontal direction is shown. The signal can beconverted by a 3-2 conversion section 174 into a 1440×480i interlacesignal. The signal is divided into two in a horizontal direction by ahorizontal filtering section 206 a. The principle of filtering is shownin parts (a) and (b) of FIG. 34. As shown in part (b), 1440 dots aredivided into odd dots 263 a and 263 b, and even dots 264 a and 264 b.Where the odd dots are labeled as Xn and the even dots are labeled asYn, a sum signal is obtained by the calculation of X+Y and adifferential signal is obtained by the calculation of X−Y. As a result,two 480P or 525i signals, each of 720×480, are obtained as shown in part(b) of FIG. 34.

Returning to FIG. 21, the number of horizontal dots of such a horizontalsum signal is reduced to 720. Since the signal is passed through thehorizontal filter, however, aliasing distortion is as low as that of anNTSC signal. A conventional reproduction apparatus reproduces only thesum signal and accordingly provides a DVD picture of exactly the samequality. The differential signal represents only a profile formed ofline-drawing. However, since the difference signal is restricted by asecond video signal output restriction information provision section 179(FIG. 60) so as not to be reproduced by an ordinary reproductionapparatus, no problem occurs. The sum signal and the differential signalare respectively encoded into MPEG streams by a first encoder 3 a and asecond encoder 3 b, and subjected to interleaving in units of aninterleave block of 1 GOP or more and MADM-multiplexed.

In the case of the movie, 3-2 conversion is performed by the 3-2conversion 174 section and MADM-recorded as an MPEG signal together withthe 3-2 conversion information 174.

In the case of the movie, 24 frames are reproduced in one second.Accordingly, a 1440×480P progressive picture is reproduced based on twointerlace signals by a 2× reproduction apparatus. The scope size of themovie is 2.35:1. The format of 1440×480P is suitable for the scope sizeof 2.35:1 in terms of the aspect ratio. Thus, a wide screen 480P iseffectively reproduced.

A wide 480i hierarchical disk 724 b is described with reference to FIG.21 above. With reference to FIG. 4, an operation for reproducing thedisk 724 b by a W-480i reproduction apparatus will be described. Whenthe disk 724 b has information recorded at 24 frames/sec., a W-480Pbasic signal 757 a and a W480P differential signal 757 b are obtained bydecoding performed by field frame conversion sections 756 a and 756 b.Each pixel is encoded by data obtained by (X+Y)/2 or (X−Y)/2.Accordingly, when a synthesis section 758 performs the calculation of(X+Y)/2+(X−Y)/2, X, i.e., data of odd pixels is obtained by thedecoding. When the synthesis section 758 performs the calculation of(X+Y)/2−(X−Y)/2, Y, i.e., data of even pixels is obtained by thedecoding. As a result, the number of pixels in the horizontal directionis doubled to 1440 pixels. In this manner, a W480P video signal 759 of1440 ×480P pixels is obtained. The W480P video signal 759 of 1440×480Ppixels is converted by a W480P-720P conversion section 760 so as to have1280 pixels in the horizontal direction using a 8/9 horizontal filter760 a, and so as to have 720 pixels in the vertical direction using a3/2 vertical filter 760 b. As a result, a 720P digital output isobtained. Thus, use of a general 720P digital interface isadvantageously allowed.

(Detailed reproduction operation: FIG. 25)

With reference to FIG. 25, an operation of a reproduction apparatus 65according to the present invention will be described in detail. FIG. 25is a block diagram of a reproduction apparatus for reproducing a 2×progressive or super-wide picture or 720P signal. A signal reproducedfrom an optical disk 1 is divided by a division section 68 into firstinterleave blocks 66 and second interleave blocks 67 each including aframe signal of 1GOP or more. Frame video signals 70 a and 70 b each of30 frames/sec. obtained as a result of MPEG extension performed by anextension section 69 are respectively divided by field division sections71 a and 71 b into odd field signals 72 a and 72 b and even fieldsignals 73 a and 73 b. Thus, 2 ch NTSC interlace signals 74 a and 74 bare output. The wide screen shown in FIG. 20 will be described later.Referring to FIG. 25, using the method described above, a 1440×960progressive picture 182 a is divided in a horizontal direction using asub band filter or wavelet conversion by a horizontal and verticaldivision section 194 in a picture division section 115. Thus, a 525progressive video signal 183 is obtained. This signal is divided into525 interlace signals 184 and recorded-as streams 188 a and the like.

Remaining interpolation information 185 is divided similarly into fourstreams 188 a, 188 d, 188 e and 188 f and recorded as interleave blocks.The maximum transfer rate of each interleave block is 8 Mbps by the DVDstandards. Accordingly, when the interpolation information is dividedinto four streams, the transfer rate is 32 Mbps. When the interpolationinformation is divided into six angles, the transfer rate is 48 Mbps.Thus, a 720P or 1050P HDTV video signal can be recorded. By theconventional reproduction apparatus, the stream 188 a is reproduced tooutput an interlace video signal 184. Regarding the streams 188 c, 188d, 188 e and 188 f, output restriction information is recorded on anoptical disk 187 by a picture processing restriction informationgeneration section 179. Therefore, the interpolation information 185,such as differential information or the like, which is not properlyviewable is prevented from inadvertently being output. By dividing thesignal in horizontal and vertical directions by the system shown in FIG.25, an optical disk compatible to both the HDTV and NTSC formats isadvantageously realized.

In FIG. 25, the interlace signal obtained by the conversion performed byan interlace conversion section 175 is output to provide a scope screen178. A 480P progressive signal is similarly output on a scope screen178. When a monitor for 720P is used, a 480P signal is converted into a720P progressive signal by a 480P/720P conversion section 176, and as aresult, is output on a 1280×720 or 1440 ×720 letter box type 720P screen177 (the picture has 1280 ×480 or 1440×480 pixels). Since the scopepicture (2.35:1) has 1128×480 pixels, a picture having a size closer tothe aspect ratio is obtained. Especially In the case of a movie, thesignal is of 24 frames/sec. and so the progressive picture istransferred at the rate of 4 Mbps. When a scope picture is recorded bythe system according to the present invention of dividing the pictureinto two screens, the transfer rate is 8 Mbps. In such a case, about 2hours of information can be recorded on a two-layer DVD. Accordingly, a720P or 480P high quality progressive picture signal for scope screencan be advantageously recorded on one DVD. On a conventional TV screen,the picture is displayed by an interlace output signal, needless to say.The present invention has an effect that a scope screen picture (2.35:1)of the movie can be output as a 480P or 720P signal.

(High resolution recording identification information)

Returning to FIG. 1, address information is output from an addresscircuit. A hierarchical recording identifier 725 includingprogressive/3D picture arrangement information is output from aprogressive/3D picture arrangement information output section 725 a.These pieces of information are recorded on the optical disk by arecording circuit 723. The progressive/3D picture arrangementinformation includes an identifier which indicates whether or not aprogressive or 3D picture is present on the optical disk, thehierarchical recording identifier 725 which indicates whether or not thesignal is up-converted when being hierarchy-encoded, or a progressive/3Dpicture arrangement table 14 shown in FIG. 17. As shown in FIG. 17, aTEXTDT file 83 includes, for each VTS, 3D pictures for the right andleft eyes and angle numbers and cell numbers in which the progressivesignal is located. Since a PGC file of each VTS includes a start addressand a termination address of each cell, the start address and thetermination address of each cell are included in the progressive/3Dpicture arrangement information. Based on the arrangement informationand identification information, the reproduction apparatus outputs aprogressive picture or a 3D picture correctly as progressive outputs orR and L outputs. When ordinary pictures of different contents from eachother are output as R and L outputs in error, the user will feeluncomfortable since the pictures for the right eye and the left eye arenot related to each other. The progressive/3D picture arrangementinformation, the progressive/3D picture identifier, or the hierarchicalrecording identifier has an effect of avoiding the output of suchunpleasant pictures. As shown in FIG. 3, when the hierarchical recordingidentifier 725 is reproduced, the control section sends an up-conversioninstruction 786 to up-convert the a 480P signal into a 720P signal bythe up-converter 738. Then, the synthesis of the 720P signal isperformed. When the hierarchical recording identifier 725 is notavailable, a 480P signal is output after performing synthesiscalculation without using the up-converter 738 as shown in FIG. 10. Inthis manner, stable picture synthesis can be performed simply byswitching the connection in accordance with whether or not theidentifier is available.

With reference to FIG. 23, a process for performing reproduction using apicture identifier 222 will be described.

From the optical disk, reproduction process control information 225 isfirst read from the management information 224. Since the information225 includes restriction information on VOB, a 0th VOB 226 a is onlyconnected to a first VOB 226 b having a main picture by an existingreproduction apparatus. Since the 0th VOB. 226 a is not connected to asecond VOB 226 a having an interpolation signal such as differentialinformation or the like, an ugly picture such as differentialinformation is prevented from being output by the existing reproductionapparatus as described above. Each VOB of the main signal has a pictureidentifier. Since the progressive identifier=1 and resolutionidentifier=00 (525) in the first VOB 226 b and the second VOB 226 c, aprogressive signal having 525 scanning lines is reproduced from aprogressive or HD reproduction apparatus.

In a picture identifier 222 of the next VOB 226 d, the progressiveidentifier=0 and the resolution identifier 219=10. This indicates thatan interlace signal having 1050 scanning lines is output and that threeVOBs 226 e, 226 f and 226 g are interpolation information. Thus, an NTSCsignal is output by a conventional reproduction apparatus, an interlacesignal having 720 horizontal pixels and 1050 vertical pixels is outputby a progressive reproduction apparatus, and a full HDTV-format signalhaving 1050 scanning lines is output by a HD reproduction apparatus. Ascan be appreciated from this, various video signals can be recorded inan interleave manner and reproduced by the picture identifier 222. Thepicture identifier 222 can be recorded in the management information224.

(2× clock and soft-decoding)

In the block diagrams shown in FIGS. 3 and 4, two MPEG decoders areused. In FIG. 18, a first MPEG signal and a second MPEG signal aresynthesized into one MPEG signal by a synthesis section 36, and a 2×clock is generated by a 2× clock generation section 37. The MPEG signalis doubled by a 2× clock-type MPEG decoder 16 c, extended, and output asR and L video signals by a division section 38. In this manner, thecircuit configuration can be simplified. This circuit configuration isrealized simply by adding a 16 MB SD-RAM to a memory 39 of the existingreproduction apparatus, without significantly raising the cost. Forsoft-decoding, when a CPU has a 2× clock, one CPU realizes simultaneousdecoding by time division. This will be described in a second example.

(Simultaneous reproduction)

With reference to FIG. 18, synchronous reproduction of two streams,which is important in decoding 3D picture data and progressive picturedata will be described. First, it is necessary to adjust vertical andhorizontal synchronization of two streams within a single line. In orderto do this, a first MPEG decoder 16 a and a second MPEG decoder 16 b arestarted substantially simultaneously by a vertical/horizontalsynchronization control section 85 a to synchronize the decoders 16 aand 16 b. Then., it is necessary that the outputs from the two decodersshould be a picture having an identical VPTS. This will be describedwith reference to the flowchart in FIG. 26 and FIG. 18. In step 241 a,the synchronization of a first decoder and a second decoder iscancelled. In step 241 b, the decoders are synchronized with each othervertically and horizontally as described above. In step 241 c, an APTSof an audio signal is read, and the APTS value is set as an initialvalue of an STC of the first decoder and an STC of the second decoder.In step 241 e, processing of the first decoder is started. In step 241f, it is checked whether or not a first VPTS has reached the initialvalue. If yes, decoding is started in step 241 g. Instep 241 h, aprocessing delay time period of the first decoder is calculated, and theVPTS of the decoder output is adjusted so that the APTS and the VPTS aresynchronized with each other. Since the second decoder is processed inthe same manner, the picture from the first decoder and the picture fromthe second decoder are synchronized with each other. Thus, the decoderoutputs, i.e., the first MPEG signal and the second MPEG signal aresynchronized within one line. Then, the synchronization on a dot-by-dotbasis is obtained by a video signal synchronization section 36 a of thesynthesis section 36. An original progressive picture is obtained evenby a sum calculation. As shown in FIG. 5, in the case where an APTS 84is read by the audio decoder 16 a and an identical APTS is set inregisters 39 a and 39 b of the STCs of the two MPEG decoders 16 a and 16b, an audio stream and the two video stream are automaticallysynchronized with one another.

In the present invention, when the buffer circuits 23 a and 23 bunderflow, either one of the pictures is disconnected, as a result ofwhich a disturbed progressive picture is output. In order to avoid this,the buffer amounts of the two buffer circuits are controlled by a bufferamount control section 23 a as shown in FIG. 2. This operation isillustrated in the flowchart shown in FIG. 27. First, in step 240 a, amaximum interleave value among the NAVI information of each disk isread, and a maximum value of 1 ILB in one main interleave block is set.The maximum value is usually 512 sectors, i.e., about 1 MB. When themaximum value is set below 1 MB by a specific format, that value is setas the maximum value. Next, when an instruction to simultaneouslyreproduce the main and sub interleave blocks is issued in step 240 b, ifthe buffer amount of the first buffer circuit 23 a is 1 ILB or less instep 240 c, an instruction to reproduce the data from the maininterleave block and transfer the data to the first buffer circuit 23 ais issued. Then, the processing goes back to steps 240 b and 240 c. Thetransfer is stopped instep 240 d when the buffer amount of the firstbuffer circuit exceeds 1ILB. Since the data in the buffer circuit 23 abecomes 1ILB or more in this manner, underflow is prevented.

In step 240 f, a maximum value of a sub interleave block of 1 ILB-Sub isset in the buffer circuit 23 b. Simultaneous reproduction is performedin step 240 g. When the data in the second buffer circuit 23 b is ½ILB-Sub or less in step 240 h, data is read into the buffer circuit instep 240 j. When the data is more than ½ ILB-Sub, the reading is stoppedin step 240 i.

As shown in part (4) of FIG. 24, the data amount of ½ ILB is sufficientin the second buffer circuit. Accordingly, the buffer amount can bereduced to half. The buffer control in FIG. 27 eliminates the underflowof the buffer circuits, thus reducing disturbance in the synthesizedpicture during reproduction.

(Required capacity of the track buffer: FIGS. 23 and 31)

First, a method for synchronizing two video streams according to thepresent invention will be described. First, as shown in FIG. 39, asystem reproduced from the optical disk is once accumulated in a trackbuffer 23 and then sent to a first video decoder 69 d and a second videodecoder 69 c. In the track of the optical disk, a first stream A and asecond stream B of the progressive signal are alternately recorded on aninterleave block-by-interleave block basis.

First, the stream A is reproduced at 2× rotation, and data accumulationin a first track buffer 23 a in the track buffer 23 is started. As shownin part (1) of FIG. 24, when t=t1 to t2, data for 1 interleave block(ILB) I1 of the first video signal for 1 interleave time T1 isaccumulated. A first track buffer data amount is increased, and becomesequal to 1 ILB at t=t2. Thus, data accumulation for 1 ILB of the firstvideo signal is completed. At t=t2, after accumulation of data for 1 ILBof the first video signal corresponding to 1 GOP or more is completed,the second video signal (stream B) is reproduced from the optical diskstarting from the interleave block I2. As shown in the solid line inpart (4) of FIG. 24, data accumulation of the second video signal in asecond track buffer 23 b is started at t=t2 and continued until t=t6.From t=t2 through t8, as shown in parts (7) and (10) of FIG. 24, thevideo presentation time stamps (VPTS) of the first video signal and thesecond video signal are synchronized and respectively sent to the firstvideo decoder 69 a and the second video decoder 69 d from the trackbuffer 23 a and the track buffer 23 b. As shown in parts (8) and (11) ofFIG. 24, the input signals are output as two pieces of video data afterbeing extended by the first and second video decoders 69 c and 69 d. Theoutput of these pieces of data starts at t=t3, which is delayed by avideo delay time period twd, which is required for MPEG extension of thedata. From t=t4 through t10, the streams A and B are synthesized into aprogressive signal by a progressive conversion section 170. Thus, aprogressive signal for one interleave block is output.

As described above, from t=t2 through t8, data for one interleave blockis input to the decoders. Accordingly, the data in the first trackbuffer 23 a and the data in the second track buffer 23 b are consumedand reduced at substantially the same rate. Therefore, as shown in part(2) of FIG. 24, the data amount in the first track buffer is reducedfrom t=t2 through t7. At t=t7, the data amount is ½ of 1 ILB. Since datareproduction for the interleave block IS starts at t=t7, the data amountincreases until t=t8, when the data amount reaches 1 ILB. Since datainput to the first decoder 69 a starts at t=t8 as at t=t2, the dataamount reduces until t=t11. Finally, the buffer memory amount becomes ½ILB.

With reference to part (4) of FIG. 24, a change in the memory amount inthe second track buffer 23 a for stream B will be described. At t=t2,input of data B1 for the interleave block I2 of stream B in the secondtrack buffer 23 b starts. At the same time, transfer of data B1 to thesecond video decoder 69 d starts. Accordingly, the buffer amount at t=t6is ½ ILB. When 2-angle recording of a progressive signal according tothe present invention is performed, it is necessary to perform a trackjump to the interleave block I5 over the interleave blocks I3 and I4from time t=6 to t=7 since there are four streams, i.e., four interleaveblocks. During the jump period 197 (tj), data input from the opticaldisk is interrupted. Thus, the buffer amount of the stream B is reduceduntil t=t8, when the buffer amount is close to zero.

Since input of data B2 of the interleave block I6 starts at t=t8, thebuffer amount starts increasing again. At t=t11, the memory amount ofthe second track buffer is ½ ILB. At t=t11, a track jump to theinterleave block I9 of A3 over the interleave blocks I7 and I8 isperformed.

The above-described operation is repeated.

Now, the minimum necessary memory capacity for a track buffer 23 (totalcapacity of the first and second track buffers 23 a and 23 b) accordingto the system of the present invention will be described. A track buffercapacity 198 indicated by the dotted line in part (4) of FIG. 24 showsthe total data amount in the first and second track buffers 23 a and 23b. A continuous reproduction is realized by setting the total capacityof a minimum 1 ILB in the track buffer.

According to the present invention, the total capacity of the trackbuffers 23 a and 23 b is set to be 1 interleave block or more forreproduction of a progressive signal. Thus, overflow and underflow ofthe track buffer are prevented.

(Method for control the system clock)

A method for switching the system clock STC between two streams will bedescribed with reference to FIG. 28. A progressive signal includes twostreams. A and B. Here, the streams of two interlace signals forming a 1ILB progressive signal are referred to as A1 and B1. As shown in part(1) of FIG. 28, data A1 for stream A is reproduced during the ½ ILB timeperiod and all the data is recorded in the buffer. Then, as shown inpart (2) of FIG. 28, data for stream B is reproduced as B1 and stored inthe buffer after A1 is reproduced. Since the data reproduced from theoptical disk is restricted with stream B (part (2) of FIG. 28) asdescribed above, the track buffer does not overflow. Stream A (part (3)of FIG. 28) or stream clock (SCR) from the track buffer for stream B isreset substantially in synchronization with the start point J of thereproduction of stream B (part (2) of FIG. 28). Since stream B is outputat the rate of 2×, the stream clock is counted at the rate of 1× asshown in part (3) of FIG. 28, i.e., at half the rate of stream B due tothe buffer. At point G, the stream clock is reset. Time VPTS2 at whichthe video signal for stream B is output from the video decoder needs tobe synchronized in consideration of the delay time period Tvd due to,for example, MPEG decoding time period. In this case, at point I (t=Ti),when the VPTS stops rising, AV synchronization control is restarted. Bychecking VPTS2 of stream B and synchronizing VPTS1 of stream A to VPTS2,synchronization is realized by one-system simple control. VPTS1 can beused additionally.

Audio data of synchronizing stream B is reproduced and the system clockis switched at point H using APTS of stream B as shown in part (4) ofFIG. 28. Regarding a sub picture signal of stream B, the STC can beswitched in a similar manner.

By using data of stream B with priority, AV synchronization is realizedwith simple control.

Since all the data in streams A1 and A2 is stored in the buffer memory,the buffer memory does not overflow. Stream B1 may possibly overflow.However, according to the present invention, the synchronization controlis performed using stream B and thus the system clock is switched tocontrol the signal flow so that VPTS2 does not exceed the VPTS2threshold level as shown in part (6) of FIG. 28. Therefore, the bufferdoes not overflow.

According to the present invention, the audio signal of stream B is usedfor audio reproduction. Therefore, the buffer amount of audio decoder isreduced to ½. Furthermore, by switching the system clock at point H(t=Th) as shown in part (4) of FIG. 28, the audio signal is reproducedsmoothly without exceeding the APTS threshold level. The sub pictureinformation is also reproduced with smooth synchronization. Accordingly,picture, audio and sub picture (subtitles or the like) signals aresynchronized, and picture and audio are reproduced seamlessly with nointerruption. The audio signal and the sub picture signals of stream Acan be omitted.

(AV synchronization: FIGS. 29, 30, 31 and 33)

AV synchronization, which is especially important for connection and thelike when a jump is performed to reproduce two or three streamssimultaneously, will be described. This is important in the presentinvention, according to which the streams of the 720P signal and the480i signal, which are significantly different from each other in thedata amount, are synchronized.

FIG. 29 is a flowchart showing the detailed process of reproduction of aprogram chain group performed by the system control section 21. As shownin FIG. 29, in steps 235 a, 235 b and 235 c, the system control section21 reads corresponding program chain information from the volumeinformation file or a program chain information table of the video file.When the program chain is not completed in step 235 d, the processingadvances to step 235 e.

In step 235 e, it is determined whether or not the current cell and theimmediately previous cell should be connected seamlessly referring toseamless connection instruction information for the cell to betransferred next in the program chain information. If seamlessconnection is necessary, the processing goes to step 235 f for seamlessconnection processing. If not, ordinary connection is performed.

In step 235 f, the mechanism control section and the signal processingsection, for example, are controlled to read DSI packets, so that VOBreproduction end time (VOB_E_PTM) in the DSI packet of the cell whichhas been transferred and VOB reproduction start time (VOB_S_PTM) in theDSI packet of the cell to be transferred next are read.

In step 235 h, “VOB reproduction end time (VOB_E_PTM) -VOB reproductionstart time (VOB_S_PTM)” is found by calculation. The resultant value issent to an STC offset synthesis section 164 in the AV synchronizationcontrol section 158 in FIG. 30 as an STC offset value between thecurrent cell and the immediately previous cell which has beentransferred.

Simultaneously, in step 235 i, VOB reproduction end time (VOB_E_PTM) istransferred to an STC switch timing control section 166 as switchingtime T4 for an STC switch 162 e.

The system control section 21 then instructs the mechanism controlsection to continue reading data up to the terminal position of thecurrent cell. Thus, the data for the current cell is transferred to thetrack buffer 23 in step 235 j. Upon completion of the transfer, theprogram chain information is read in step 235 c.

If it is determined the seamless connection is not necessary in step 235e, the data is transferred to the track buffer 23 up to the end of thesystem stream, and then program chain information is read in step 235 c.

Hereinafter, two examples of a method for AV synchronization control forseamless connection to perform seamless reproduction will be described.In other words, the AV synchronization control section 158 shown inFIGS. 2 and 31 will be described in detail.

Referring to FIG. 31, a system decoder 161, an audio decoder 160, videodecoders 69 a and 69 d, and a sub picture decoder 159 are allsynchronized to a system time clock given by the AV synchronizationcontrol section in FIG. 30 to process the data in the system stream.

Regarding a first method, the AV synchronization control section 158will be described with reference to FIG. 30.

In FIG. 30, the AV synchronization control section includes STC switches162 a, 162 b, 162 a and 162 d, an STC 163, an STC offset synthesissection 164, an STC setting section 165 and an STC switch timing controlsection 166.

The STC switches 162 a, 162 b, 162 a, 162 d and 162 e switch between anoutput value of the STC 163 and an output value of the STC offsetsynthesis section 164 as a reference clock to be provided to the systemdecoder 161, the audio decoder 160, the main video decoder 69 c, the subvideo decoder 69 d and the sub picture decoder 159, respectively.

The STC 163 is a reference clock of the entire MPEG decoder shown inFIG. 31 in ordinary reproduction.

The STC offset synthesis section 164 continues outputting a valueobtained by subtracting the STC offset value provided by the systemcontrol section from the value of the STC 163.

The STC setting section 165 sets an STC initial value given by thesystem control section or an STC offset synthesis value given by the STCoffset synthesis section 164 in the STC 163 at the timing given by theSTC switch timing control section 166.

The STC switch timing control section 166 controls the STC switches 162a through 162 e and the STC setting section 165 based on STC switchtiming information given by the system control section, the STC 163, andthe STC offset synthesis value given by the STC offset synthesis section164.

The STC offset value is an offset value used for changing the STC valuewhen system stream #1 and system stream #2 having different STC initialvalues are continuously reproduced.

The STC offset value is specifically obtained by subtracting the “VOBreproduction start time (VOB_S_PTM)” described in the DSI of systemstream #2 to be reproduced next from the “VOB reproduction end time(VOB_E_PTM)” described in the DSI packet of system stream #1 reproducedfirst. The information regarding the display of such a value ispre-calculated by reading data from the optical disk in FIG. 5 by thesystem control section 167 when the data is input to the track buffer23.

The calculated offset value is supplied to the STC offset synthesissection 164 before the last pack of system stream #1 is input to thesystem decoder 161.

Except for seamless connection control, the data decoding processingsection 165 in FIG. 5 operates as an MPEG decoder. The STC offset valuegiven by the system control section 21 is 0 or an arbitrary value. TheSTC switches 162 a through 162 e are always selected to be connected tothe STC 163.

With reference to the flowchart in FIG. 33, switching of the STCswitches 162 a through 162 e in the connection part of the systemcontrol section and an operation of the STC 163 when two system streamshaving non-continuous STC values, such as system streams #1 and #2, arecontinuously input to the system decoder 161, will be described.

The SCR, APTS, VPTS and VDTS of the system streams #1 and #2 to be inputwill not be described.

It is assumed that in the STC 163, an initial STC value corresponding tosystem stream #1 which is being reproduced is set by the STC settingsection 165, and the value is sequentially counted up in accordance withthe reproduction. The system control section 21 (FIG. 31) calculates theSTC offset value by the above-described method and sets this value inthe STC offset synthesis section 164 before the last pack of systemstream #1 is input to the decoder buffer. The STC offset synthesissection 164 continues outputting a value obtained by subtracting the STCoffset value from the value of the STC 163 (step 168 a).

The STC switch timing control section 166 obtains time T1, at which thelast pack of system stream #1 reproduced first is input to the decoderbuffer, and switches the STC switch 162 a to the output side of the STCoffset synthesis section 164 at time T1 (step 168 b).

Thereafter, the STC value referred to by the system decoder 161 isprovided with an output from the STC offset synthesis section 164. Thetransfer timing of system stream #2 to the system decoder 161 isdetermined by the-SCR described in the pack header of system stream #2.

Next, the STC switch timing control section 166 obtains time T2, atwhich the reproduction of the last audio frame of system stream #1reproduced first is terminated, and switches the STC switch 162 b to theoutput side of the STC offset synthesis section 164 at time T2 (step 168c). A method for obtaining time T2 will be described later.

Thereafter, the STC value referred to by the audio decoder 160 isprovided with an output from the STC offset synthesis section 164. Theaudio output timing of system stream #2 is determined by the APTSdescribed in the audio packet of system stream #2.

Next, the STC switch timing control section 166 obtains time T3 and T3′,at which the decoding of the last video frame of the main signal and thesub signal of system stream #1 reproduced first is terminated, andswitches the STC switches 162 a and 162 d to the output side of the STCoffset synthesis section 164 at time T3 and T3′ (step 168 d). A methodfor obtaining time T3 will be described later. Thereafter, the STC valuereferred to by the video decoders 69 a and 69 d is provided with anoutput from the STC offset synthesis section 164. The video decodingtiming of system stream #2 is determined by the VPTS described in thevideo packet of system stream #2.

Next, the STC switch timing control section 166 obtains time T4, atwhich the reproduction output of the last video frame of system stream#1 reproduced first is terminated, and switches the STC switch 162 e tothe output side of the STC offset synthesis section 164 at time T4 (step168 e). A method for obtaining time T4 will be described later.

Thereafter, the STC value referred to by the video output switch 169 andthe sub picture decoder 159 is provided with an output from the STCoffset synthesis section 164. The video output timing and sub pictureoutput timing of system stream #2 are determined by the VPTS and SPTSdescribed in the video packet and the sub picture packet of systemstream #2.

When switching of the STC switches 162 a through 162 e is completed, theSTC setting section 165 sets the value given by the STC offset synthesissection 164 in the STC 162 (step 168 f) (referred to as “reloading ofthe STC 163) and switches all the switches 162 a through 162 e to beconnected to the STC 163 (step 168 g).

Thereafter, the STC value referred to by the audio decoder 160, thevideo decoders 69 a and 69 d, the video output switch 169 and the subpicture decoder 159 is provided with an output from the STC 163, and theoperation returns to the ordinary operation.

Now, two means for obtaining time T1 through T4 for switching the STCwill be described.

According to specific means, information representing time T1 throughT4, which can be easily calculated when the streams are created, isrecorded on the disk. The system control section 21 reads theinformation and sends the information to the STC switch timing controlsection 166.

Especially for T4, “VOB reproduction end time (VOB_E_PTM)” described inthe DSI used for obtaining the STC offset is used as it is.

On the disk, the value obtained based on the STC value used in systemstream #1 reproduced first is described, and the STC switch timingcontrol section 166 switches the STC switches 162 a through 162 e at themoment the value of the STC 163 becomes time T1 through T4.

EXAMPLE 2

In the first example, an example of application of a system forreproducing a plurality of streams in synchronization according to thepresent invention is described in detail. In a second example, thissystem is applied to a reproduction control system for reproducing twostreams seamlessly. In the case of recording an MPEG signal, editing isconventionally performed on a GOP-by-GOP basis in general, and it isdifficult by conventional methods to perform editing on a frame-by-framebasis. By using an MSS system according to the present invention,substantial frame-based editing is realized.

It is important to synchronize the timing of the video signal and theaudio signal at the point of connection. Specific synchronizationsystems will be described in third through ninth examples.

The reproduction control system according to the present invention canbe applied as follows to connect two streams while switching the twostreams on a frame-by-frame basis seamlessly. As shown in FIG. 10,editing data 761 including synthesis information of a 28P zoominstruction signal and the like is processed by an editing dataprocessing section 762 and sent to a switching synthesis section 763.The data is then switched/synthesized and output from a switchingsynthesis signal output section 764. Thus, two MPEG video signals can beswitched and connected seamlessly at an arbitrary point other than theborders between GOPs.

A system for synthesizing two pictures at an arbitrary point based on aninstruction signal will be described later with reference to FIG. 22.

In a simple switching mode, two pictures are simply switched over frameby frame at an editing point as follows. Stream “a” and stream “b” areswitched over at an editing point tc, and the resultant stream is outputseamlessly. In a synthesis switching mode (such as a wipe), stream “a”and stream “b” are switched after being synthesized from a start pointts to a termination point te. As shown in FIGS. 6 and 58, in mode 1,switching is performed from left to right; in mode 2, from center toperiphery; in mode 3, top to bottom; and in mode 4, in a mosaic manner.FIG. 6 is a simplified block diagram; and FIG. 58 is a detailed blockdiagram.

In FIG. 6, reproduction means 778, a division section 734, a VTSsynchronization section 780, and MPEG decoders 728 and 730 have exactlythe same structure as that of those in the 480P reproduction apparatusshown in FIG. 3, and thus the 480P reproduction apparatus can be used.With reference to FIG. 6, when an identification information processingsection 766 detects reproduction control information 766 a, two videostreams are sent to the switching synthesis section 763, and the firststream is switched to the second stream seamlessly at the connectionpoint ts as described above.

With reference to FIG. 58, when a 720P/480P hierarchical recordingidentifier 725 having a high resolution signal described in the firstexample is detected, the synthesis section 732 a performs calculationand outputs a high resolution signal such as a 480P or 720P signal.

When a 3D recording identifier 766 a is detected, a 3D signal processingsection 770 generates a 3D video signal having a right-eye video signaland a left-eye video signal alternately interleaved, and outputs thesignal.

In this manner, the MSS system shown in FIG. 58, which uses two MPEGdecoders or an MPEG decoder for decoding two streams simultaneously, canbe used to provide three functions, i.e., reproduction control withframe-based editing, reproduction of a high resolution signal, andreproduction of a 3D video signal.

FIG. 11 shows a specific example of reproduction control information765. The reproduction control information 765 includes a switching pointnumber S 766, a synthesis mode identifier 767, a first stream switchingstart address ts 768, a first stream switching termination address te2769, a second stream GOP start address tsG 790, a second streamswitching start address ts2 771, and a second stream switchingtermination address te2, 772 in the order of the address used.

Specifically, when the switching point number S=1, as shown in part (9)of FIG. 12, a picture synthesis identifier 767 is not present or is 0.Accordingly, the first stream is simply switched into the second streamat the switching address of ts1-1. When S=2, as shown in part (10) ofFIG. 12, switching is started at ts1, and the two pictures of the firststream and the second stream are synthesized into one picture untilt=te1. At t=te1, the first stream is completely switched into the secondstream.

With reference to the flowchart of FIG. 13, a process for reproductionperformed based on the reproduction control information will bedescribed.

With reference to part (1) of FIG. 12, a GOP 781 a of the first streamand a GOP 781 b of the second stream may be at discrete positions on anoptical disk without the streams being moved or rewritten. In this case,the time period for rewriting can be saved. With reference to part (3)of FIG. 12, since data can be recorded on a DVD-RAM disk and the likeafter edited, streams are recorded on a GOP-by-GOP basis; i.e., a GOP781 e and a GOP 781 f both including a frame at the editing point arerecorded adjacent to each other. In this case, the editing point isallowed to be changed in one GOP later. When two streams are synthesizedin a wipe manner, the picture of the first stream after the editingpoint is required. Thus, the structure shown in part (3) of FIG. 12 isrequired.

When S=0, i.e., when synthesis of two pictures is not performed, thedata after the switching point ts1 of a GOP 781 a is not required andthus is deleted as shown in part (2) of FIG. 12 to exclude the redundantportion. Thus, the recording efficiency is raised. However, a GOP 781 d,which has the IN point, includes an i (intra) frame (i.e., basic frame)at the beginning, which cannot be deleted. Thus, a redundant portion 783is generated.

As shown in step 792 a in FIG. 59, 1GOP includes about 15 actual frames.When the B frames present before the IN point are deleted as in step 792b, the number of frames is reduced from 12 in step 792 a (redundantportion 783 f) to 3 in step 792 a (redundant portion 783 g). Theredundant portion 783 g is about ¼ of the redundant portion 783 f, andthus the recording efficiency is raised.

When this portion is reproduced, a B frame deletion identifier isdetected in step 792 f. The number of frames are calculated with apremise that the B frames are not recorded. Since an MPEG signal isdecoded with only I, P frames, the frames are decoded one after anotherin step 792 g. The frame having the IN point (I.e., t=ts2) is obtainedby decoding and output. In this case, the number of frames to beprocessed is only three. Accordingly, the intended IN point can bereproduced in a ¼ time period. In this example, the redundant portion is⅛ second long. As can be appreciated from FIG. 59, ts2 is in the 14thframe in the worst case. In this case, the redundant portion is fiveframes long of I, P, P, P, B, i.e., ⅓×½=⅙ second long. That is, thelongest possible redundant portion is about 0.18 second. At least such atime period is necessary to reproduce the IN point. Even if about fivecut portions are included in one second, each cut portion is presentevery 0.2 second. By deleting the B frames by this system, theframe-based reproduction is realized even if there are five cut portionsin one second. This means that this system is usable in the standardediting with no problem.

A method for generating reproduction control information will bedescribed. Where the final GOP before the OUT point is defined as afirst GOP and the first GOP after the IN point is defined as a secondGOP, reproduction control can be performed simply by recording the timeof the beginning of the second GOP as the switching point ts2 and thetime of the switching point. Alternatively, the number of frames fromthe beginning of the second GOP to the switching point can be recorded.

When such reproduction control information is reproduced by areproduction apparatus, the frame at the switching point is decodedwithout processing the B frames (pictures) among I, B, B, B, P, B, B, B,P, B, B, B as shown in step 792 f in FIG. 59. In other words, only I, P,P are decoded. In this manner, the picture at the IN point is obtainedin a ¼ time period as described above. Even if there are five frameswitching points in one second, the redundant time period is 0.18second, which is shorter than the pitch of the cut portion (0.2 second).Thus, the data can be reproduced seamlessly at ail the switching points.

In order to realize synchronization, reproduction control information isobtained by calculating the number of frames (pictures) existing betweenthe second GOP and the switching point based on the time of thebeginning of the second GOP and the time of the switching point. Whenthe B frames which are not necessary are deleted, the correction isperformed in consideration of the deleted frames referring to anunnecessary frame deletion identifier. Then, it can be found how manyframes should exist between the start of reproduction of the second GOP,the IN point of the second GOP, and the OUT point of the first GOP, inorder to synchronize the OUT point of the first GOP and the IN point ofthe second GOP.

When the number of frames existing between the beginning of the secondGOP and the switching point is recorded as the reproduction controlinformation, correction in consideration of the unnecessary frames suchas the B frames can be performed in order to find the time to startdecoding the second GOP to realize the synchronization.

Another recordable reproduction control information can be decodingstart timing information which indicates a specified point in the firstGOP at which decoding of the second GOP should be started in order tomatch the switching point of the first GOP and the switching point ofthe second GOP.

In such a case, the switching points can be synchronized using only thereproduction control information, without performing any specialcalculation by the reproduction apparatus.

The redundant portion 783 includes one I frame, a plurality of P framesand a plurality of B frames. By decoding these frames to create a frameimmediately before the frame corresponding the final editing point(i.e., intraframe), the recording efficiency can further be raised. Inthe case of a DVD-RAM disk, the overall editing structure is found astoc by recording the overall reproduction control information 765 andlimited reproduction control information 765 a on only the switchingpoints, at two points, i.e., at the beginning of the recording data andimmediately before the editing point as shown in parts (1), (2) and (3)of FIG. 12. Before the editing point, limited reproduction controlinformation 765 a on each separate editing point, for example, S=1 canbe recorded among the overall reproduction control information. In thiscase, reproduction control for special reproduction is advantageouslystabilized.

A process for reproduction control will be described. First, in step 774a, reproduction control information is read. In step 774 b, thereproduction switching point number S is set to 0. In step 774 a, S isincremented by one. The decoding start position of the second streamneeds to be specified. In step 774 d, it is checked whether or not t=ta,where t is the system clock or the VPTS of the first stream and ta isthe decoding start position information. When t=ta, i.e., when the VPTSof the second stream reaches ta as shown in part (5) of FIG. 12, theprocessing advances to step 774 e, where MPEG-decoding of the GOP of thesecond stream is started as shown in part (6) of FIG. 12. In step 774 f,it is checked whether or not t=ts1. As shown in parts (5) and (6) ofFIG. 12, the switching point ts2 comes after the time corresponding tothe value of (ts1-ta)=(ts2-tsG) passes. As shown by the expression andparts (5) and (6) of FIG. 12, the data at ts1 of the first stream anddata at ts2 of the second stream are MPEG-decoded at the same time andoutput from the decoder. Parts (8) and (9) of FIG. 12 show the statewhere the two streams are synchronized. Specific methods forsynchronization will be described in detail in the third through ninthexamples. It is appreciated from the reproduction control informationthat ts1 and ts2 are frame editing points of the two pictures. In step774 g, it is determined whether or not there is a picture synthesisidentifier 767. If the picture synthesis identifier 767 does not exist,the processing advances to step 774 h, where the first stream isswitched into the second stream by the switching synthesis section 763(FIG. 6) at the position of t=ts1 as shown in part (10) of FIG. 12.Since t=ts1 is the OUT point in actuality, the frame of the first streamat t=ts1 is not output. Since t=ts2 is the IN point, the frame of thesecond stream at this point is output. Since the frame information onthe first stream is not necessary at t=ts2, recording of suchinformation can be omitted when S=0, i.e., when no picture synthesis isperformed. In this case, the recording efficiency is raised by one Pframe. Thus, reproduction control of the switching point for S=1 (simpleswitching mode) is completed. The processing returns to step 774 c anddeals with the switching point of S=2.

If n=0 in step 774P, it is converted into n=1. In other words, when thesecond stream is decoded by the second MPEG decoder 730, the secondstream, i.e., the MPEG signal to be switched next is decoded by thefirst MPEG decoder 728. A different MPEG decoder is used from the caseof S=1.

The second stream is decoded as shown in FIG. 56. Input second streams781 a, 781 b, 781 g, 781 h and 781 i are sent by the division section734 to the first MPEG decoder 728 and the second MPEG decoder 730alternately and decoded into video signals 788 a and 788 b. The videosignals 788 a and 788 b are synthesized into one stream by the switchingsection 763. As is clear from the figure, the output from the first MPEGdecoder 728 is stopped after the decoded video signal 788 a is output.That is, the picture is frozen. This occurs since non-continuous datacannot be normally decoded. The MPEG decoding is resumed by performingresetting 790 of registers and the like. The second MPEG decoder isprocessed similarly. Conventionally, since only one MPEG decoder is usedto perform seamless reproduction, various complicated pre-processingneeds to be performed at the time of recording. According to the presentinvention, such various complicated processing is not necessary at thetime of recording for the following reason. Since even if one of theMPEG decoders is stopped, the other MPEG decoder can be used, even MPEGdata which could not otherwise be connected seamlessly can be reproducedseamlessly. While the second MPEG decoder operates, the first MPEGdecoder is processed to resume operating. In this manner, seamlessreproduction can be continued endlessly by switching the two MPEGdecoders. Even frame-based editing of MPEG signals, which conventionallyrequires complicated processing, can be realized without complicatedprocessing by using reproduction control according to the presentinvention.

The system according to the present invention does not require the stepof decoding and re-encoding MPEG data and thus provides a significanteffect that the picture quality is not deteriorated.

The process for seamless reproduction can be summarized as follows. Aplurality of streams are divided and alternately decoded by two MPEGdecoders. An output from one of the MPEG decoders is switched into anoutput from the other MPEG decoder at a switching point. While the otherMPEG decoder is outputting data, the first MPEG decoder is reset todecode the next stream. At the next switching point, the output from thesecond MPEG decoder is switched into the output from the first MPEGdecoder.

Thus, continuous frame-based reproduction is realized.

(Synthesis switching mode)

Returning to step 774 g of FIG. 13, a process for switching two streamswhile synthesizing the two streams into one picture in, for example, awipe manner, by the synthesis signal output section 764 (shown in FIG.6) will be described. In this case, the picture synthesis identifier 767is not 0. In step 774 i, the first stream is switched into the secondstream as shown in part (11) of FIG. 12 while the two streams aresynthesized from t=ts1. The switching is continued until the timebecomes t=te1 or t=te2 in step 774 j. The switching is completed in step774 k. Since the decoding of the first stream is stopped at t=ts1,wasteful decoding of data is prevented. The switching is performed fromleft to right as in the screen 782 a in mode 1, from center to peripheryas in the screen 782 b in mode 2, top to bottom as in the screen 782 ain mode 3, and in a mosaic manner as in the screen 782 d in mode 4.

FIG. 12 shows an example in which the original time stamp is notchanged. When the recording is performed using a DVD-RAM disk or thelike where the time of the switching point of the first stream matchesthe time of the switching point of the second stream, the time stamp canbe changed and thus the structure is further simplified. The time stampcan be changed at the time of recording by setting ts1=ts2. In thiscase, the first stream can be recorded up to te1. Time stamp tsG of thesecond stream is changed to ts1−(ts2−tsG). A time stamp having a smallervalue than ts1 is assigned. Time ta is identical with tsG. Accordingly,when reproduction is performed as shown in part (6) of FIG. 12, tsG,i.e., the GOP start address 770 a of the second stream is pre-read basedon the reproduction control information 765 b shown in FIG. 14, anddecoding is started at tsG.

When a system of changing the time stamp is used, the time stamp of aGOP 761 e and the time stamp of a GOP 781 f in part (3) of FIG. 12 arechanged in the order and also overlapped with each other. When such astream is reproduced, the reproduction apparatus malfunctions. Accordingto the present invention as shown in FIG. 14, the GOP start address 770a of the second stream is recorded in the reproduction controlinformation 765 b. Thus, the positions of the time stamps which arechanged in the order and also overlapped are found in advance.Therefore, malfunction can be advantageously prevented even whilefast-forwarding. Use of the GOP start address 770 a further secures theswitching into the second stream at the editing point.

When the reproduction is performed as shown in part (6) of FIG. 12, tsG,i.e., the GOP leading address 770 a of the second stream is pre-readbased on the reproduction control information shown in FIG. 14, anddecoding is started at tsG.

Alternatively, when information on the editing point ts1 is available,the leading address tsG of the GOP including the editing point ispre-read and the data on the GOP 781 f of the second stream isMPEG-decoded at t=tsG. This provides an advantage that the editing pointof the first stream and the editing point of the second streamautomatically match at ts1.

By recording data with the time stamps being changed as above, thestructure and operation can be significantly simplified.

The above-mentioned second recording method will be described withreference to FIGS. 15 and 16.

With reference to FIGS. 6 and 16, an editing/reproduction controlinformation generation program will be described. As shown in FIG. 6,editing information 780 including the IN and OUT points (editing points)is input manually or based on data and sequentially converted intoreproduction control information 765 by a reproduction controlinformation generation section 789. The reproduction control information765 is once stored in a memory 779 and recorded by recording means 777of the RAM disk 724 from the memory 779 either after all the editingoperation is completed or immediately before the disk is removed fromthe apparatus.

A process performed by the reproduction control information generationsection 789 will be described in detail with reference to FIG. 16.

In step 785 a, the editing information 780 is input manually or based ondata sequentially. S represents the number of the editing point, and Grepresents modes 1 through 4 in which two pictures are switched whilebeing synthesized by wipe or the like. ts0 represents the start point ofthe first stream, ts1 represents the OUT point of the first stream, te1represents the OUT synthesis completion point of the first stream, ts2represents is the IN point of the second stream, te2 represents the INsynthesis completion point of the second stream, and tL2 is the OUTpoint of the second stream.

In step 785 b, S=0 is set. S is incremented by one (step 785 c), and ts0and ts1 are read (step 785 d). In step 785 e, it is checked whether ornot G exists. If G does not exist, there is no need to synthesizepictures, and thus in step 785 f, the first stream (S) is recorded onthe optical disk 724 from ts0 to ts1. The IN point ts2 of the secondstream (S+1) is read, and the processing goes into a time stampconversion routine (step 785 g). In the time stamp conversion routine,the leading time stamp tsG of the leading GOP including informationcorresponding to the frame corresponding to ts2, and the time stamp ts2corresponding to the final frame among the frames corresponding to theleading GOP are obtained. The value of the time stamp is reduced by theamount corresponding to (ts2−ts1) from tsG to tL2 of all the recordingdata of the second stream, thereby generating a new time stamp. In step785 i, original address tsG to tL2 (tf2) of the second stream isreplaced with the new time stamp, and the information corresponding tothe frame immediately following the frame corresponding to ts1 (OUTpoint of the first stream), is rewritten by the information obtained bythe new time stamp.

In step 785 w, the address tsG of the leading GOP of the second streamis used to perform the calculation of ta=ts1−(ts2−tsG) so as to find thetime period ta prior to the decoding of the second stream. The resultanttime period is added to the limited reproduction control information 765a and the reproduction control information 765. In step 785 j, limitedreproduction control information 765 a only on the S number is recordedin the second half of the first stream (S) as shown in part (2) of FIG.12. If S is terminated in step 785 j, the processing advances to step785 m; and if S is not terminated in step 785 j, the processing returnsto step 785 a to repeat the steps. In step 785 m, the reproductioncontrol information 765 at all the editing points stored in the memory779 is recorded in the reproduction control information recordingsection of the optical disk, where management information such as TOCand the like are recorded, as shown in part (3) of FIG. 12. Thus, theediting/reproduction control information generation program iscompleted.

If a synthesis identifier G exists, in step 785 n, te1 (OUT synthesiscompletion point of the first stream) is read. In step 785 p, thecalculation of tG=te1−ts1 is performed. If tG<tGmax in step 785 q, theprocessing advances to step 785 r. If tG>tGmax, the synthesis timeperiod of the two streams is very long and exceeds the capacity of thebuffer of the reproduction apparatus. In step 785 v, an error messagethat “decrease te1” is issued. When te1 is changed in step 785 w, theprocessing returns to step 785 n, and te1 is decreased to be within thecapacity.

Thus, the synthesis of the connection point is within the capacity.Accordingly, the processing returns to step 785 r, where the firststream is recorded on the optical disk from tsG to tel. In step 785 s,ts2 of the second stream is read; and in step 785 t, the time stampconversion processing routine is performed in a similar manner to thestep 785 g, to convert the time stamp. In step 785 u, the originaladdress tsG to tf2 (tL2) of the second stream is replaced with the newtime stamp, and the information corresponding to the frame immediatelyfollowing the frame corresponding te1 of the first stream is rewrittenwith the information obtained by the new stamp. Time ta is recorded inthe memory. In step 785 j, the limited reproduction control information765 a is recorded. In step 785 k, it is checked whether or notprocessing for S=1 through S=n is completed. In step 785 m, the overallreproduction control information 765 is recorded on the optical disk.Thus, all the operations are completed.

In this manner, the first stream is recorded up to a connection point,and a portion after the connection point is rewritten with a time stamphaving a smaller value of that of the connection point, so that theconnection of the first stream and the connection point of the secondstream match each other. Thus, the reproduction apparatus can output avideo signal including the two streams connected to each other on aframe-by-frame basis.

In this case, two MPEG decoders are required. The recording andreproduction apparatus shown in FIG. 6 includes the MPEG encoder 791 forMPEG-encoding the input video signal. Since an MPEG encoder is neverused at the same time as an MPEG decoder and also the MPEG encoder has aprocessing capability of twice or more of the MPEG decoder, one MPEGprocessing section has a function corresponding to one encoder or twodecoders.

Accordingly, when the present invention is applied to the recording andreproduction apparatus including an MPEG encoder, the frame-basedediting is realized without adding any element.

Soft-encoding/decoding performed by a CPU will be described. Thecapability of a CPU for encoding one stream corresponds to thecapability for decoding two streams. As shown in FIG. 57, two streamscan be decoded simultaneously or in a time division manner. Thus, thevirtual frame-based editing according to the present invention isrealized without raising the processing capability of the CPU.

With reference to the flowchart of FIG. 57, a process for performingencoding/decoding and recording/reproduction using a CPU will bedescribed. In step 792 a of encode recording, data of m=1 to final isinput. In step 792 a and 792 d, the m'th video signal is input. The m'thvideo signal is encoded to create an m'th stream (step 792 e) andrecorded on an optical disk (step 792 f). If m is not final in step 792g, the processing returns to step 792 c; if m is final in step 792 g,the recording is terminated (step 792 h). In step 792 i of recordingreproduction control information, editing is performed on aframe-by-frame basis, and the reproduction control information and theabove-mentioned various identifiers are recorded on the optical disk(FIGS. 6 and 58).

Reproduction is performed as follows. The reproduction control program792 j is started. Cut point S is reproduced from 1 to final (steps 774 band 774 c). As shown in steps 774 m and 774 e, two streams areMPEG-decoded simultaneously or in a time division manner at theframe-edited point. In step 774 h, one decoded stream is switched to theother decoded stream at t=ts. This operation is repeated until the finalS is processed in step 774 r.

The CPU has a capability of MPEG-encoding one stream. This means the CPUhas a capability of MPEG-decoding two or three streams. Accordingly, oneCPU can MPEG-encode one stream, performs frame-based editing,MPEG-decodes two streams, and outputs the connected stream seamlessly.The present invention has an effect that a part of the capability of theCPU which is not conventionally used is effectively used.

EXAMPLE 3

The MADM system according to the present invention simultaneouslyreproduce a plurality of streams. Synchronization systems are important.

In the first example, recording and reproduction of high resolutionvideo signals such as 480P and 720P signals are described. In the secondexample, basic AV synchronization systems for reproduction control usingvirtual frame-based editing are described. In the third through ninthexamples, various methods of synchronization will be described in moredetail.

In the third example, an operation of a reproduction apparatus forreading data from an optical disk having three compression video signalsto be reproduced simultaneously, and extending and reproducing the threecompression video signals simultaneously by an AV synchronization systemwill be described.

FIG. 37 shows a data structure of an optical disk used in the opticaldisk reproduction apparatus in the third example.

Video signals A, B and C are MPEG-compressed to obtain compression videostreams A, B and C.

The compression video streams A, B and C are each packeted in units of 2kB into video packets. A packet header of each packet includes a streamID for indicating which one of the compression video streams A through Cis stored. When the packet stores a leading part of the video frame, thepacket header also includes a VPTS (video presentation time stamp) asvideo reproduction time information indicating the time to reproduce theframe. In the third example, an NTSC signal is used as the video signal,and the video frame cycle is about 33 msec.

On the optical disk, video packets created in the above-described mannerare grouped into, for example, compression video signals A-1, B-1 andC-1 each including an appropriate number of packets based on the datastored, and multiplexed.

FIG. 35 is a block diagram of an optical disk reproduction apparatus inthe third example.

In FIG. 35, the optical disk reproduction apparatus includes an opticaldisk 501 described above, an optical pickup 502 for reading data fromthe optical disk 501, signal processing means 503 for performing aseries of signal processing such as binarization, demodulation, anderror correction to the signal read by the optical pickup 502, a buffermemory 504 for temporarily storing the data output from the signalprocessing means 503, division means 505 for dividing the data read fromthe buffer memory 504 into compression video signals, and reference timesignal generation means 506 for generating a reference time signal 506including a counter (not shown) for counting 90 kHz clocks. Referencenumerals 510, 520 and 530 represent buffer memories for temporarilystoring the compression video signals divided by the division means 505.Reference numerals 511, 521 and 531 represent video decoders forextending and reproducing the compression video signals. Referencenumerals 512, 522 and 532 represent monitors for displaying the videosignals.

FIG. 36 shows the structure of each of the video decoders 511, 521 and531.

As shown in FIG. 36, the video decoder includes VPTS detection means 601for detecting a VPTS stored in the packet header of the video packet,video extension means 602 for MPEG-extending the compression videostream, and video reproduction timing control means 603 for comparingthe reference time signal and the VPTS and skipping or repeating thevideo reproduction on a frame-by-frame basis when the comparison resultexceeds the threshold value.

The optical disk reproduction apparatus shown in FIG. 35 operates in thefollowing manner.

The optical pickup 502 is focus-controlled or tracking-controlled byservo means (not shown) to read a signal from the optical disk 501 andoutputs the signal to the signal processing means 503. The signalprocessing means 503 subjects the signal to a series of processingsincluding binarization, demodulation, error correction and the like.Then, the signal processing means 503 stores the resultant signal in thebuffer memory 504 as digital data.

The buffer memory 504 functions so that, even when the data supply fromthe optical disk 501 is temporarily stopped by, e.g., a wait state, thedata supply to the subsequent-stage sections is not stopped.

The data read from the buffer memory 504 is divided into compressionvideo signals A through C by the division means 505 and output. Thedivision means identifies which of the compression video streams Athrough C is stored in each packet using the stream ID in the packetheader of the packeted data, and determines the destination based on theidentification result.

The divided compression video signals are respectively stored in buffermemories 510 through 530.

The buffer memories 510 through 530 act to continuously supply data tothe video decoders 511 through 531.

The video decoders 511 through 531 read data from the buffer memories510 through 530 respectively, extend the compression video signals, andoutput the signals as video signals to the monitors 512 through 532respectively.

With reference to FIG. 36, operation of the video decoders 511 through531 will be described.

The compression video signal read from the buffer memory is input to theVPTS detection means 601 and the video extension means 602.

The video extension means 602 MPEG-extends the compression video streamand outputs the video signal.

The VPTS detection means 601 detects the VPTS of the packet header andoutputs the VPTS.

The video reproduction timing control means 603 receives the videosignal output from the video extension means 602, a reference timesignal and the VPTS output from the VPTS detection means 601, andcompares the reference time signal and the VPTS. When the differencebetween the two exceeds the threshold value, the video reproductiontiming is controlled so that the difference between the VPTS and thereference time signal is equal to or less than the threshold value.

In the third example, 33 msec is used as the threshold value for videoreproduction. The video reproduction timing control means 603 performsthe following.

(reference time signal-VPTS)>33 msec.:1 frame is skipped.

(reference time signal-VPTS)<−33 msec.:1 frame is repeated.

In the third example, due to the precision error of the crystaloscillator used in the reference time signal generation means 506 andthe video decoders 511 through 531, the video decoders 511 and 531 areslower and the video decoder 521 is faster in terms of extension andreproduction relative to the reference time signal. Unless reproductiontiming is adjusted, the reproduced video signals are out ofsynchronization.

FIG. 38 is a timing diagram of video reproduction in the third example.Part (a) of FIG. 38 shows the reference time signal with respect toreproduction time t. Part (b) shows the VPTS#A, which is a VPTS of thecompression video signal A to be extended by the video decoder 511, part(c) shows the VPTS#B, which is a VPTS of the compression video signal Bto be extended by the video decoder 521, and part (d) shows the VPTS#C,which is a VPTS of the compression video signal C to be extended by thevideo decoder 531.

The video decoder 511 continues extension and reproduction of thecompression video signal A, and the difference between the VPTS#A andthe reference time signal exceeds 33 msec. as the threshold value at T1.Accordingly, the video reproduction timing control means of the videodecoder 511 skips one frame, which is originally to be reproduced, toadjust the reproduction timing so that the difference between the VPTS#Aand the reference time signal is equal to or less than the thresholdvalue.

The video decoder 521 continues extension and reproduction of thecompression video signal B, and the difference between the VPTS#B andthe reference time signal exceeds −33 msec. as the threshold value atT2. Accordingly, the video reproduction timing control means of thevideo decoder 521 reproduces one frame in repetition, which has beenalready reproduced, to adjust the reproduction timing so that thedifference between the VPTS#B and the reference time signal is equal toor less than the threshold value.

Similarly, the video decoder 531 continues extension and reproduction ofthe compression video signal C, and the difference between the VPTS#Cand the reference time signal exceeds 33 msec. as the threshold value atT3. Accordingly, the video reproduction timing control means of thevideo decoder 531 skips one frame, which is originally to be reproduced,to adjust the reproduction timing so that the difference between theVPTS#C and the reference time signal is equal to or less than thethreshold value.

As described above, in the third example, when the difference betweenthe reference time signal and the VPTS detected by each video decoderexceeds the threshold value, the video reproduction timing control meansof each video decoder performs adjustment so that difference between thereference time signal and the VPTS does not exceed the threshold value.In this manner, the video signals reproduced by video decoders can besynchronized with one another.

EXAMPLE 4

The fourth example relates to a reproduction apparatus for adjusting areference time signal using audio reproduction time informationindicating the time to reproduce the audio signal and synchronizes aplurality of video signals based on the reference time signal.

FIG. 41 shows a data structure of an optical disk used in an opticaldisk reproduction apparatus in the fourth example. The optical diskincludes compression audio data in addition to the data included in theoptical disk used in the third example.

An audio signal is audio-framed in units of 32 msec. for compression toobtain a compression audio stream. The audio stream is packeted in unitsof 2 kB into audio packets and recorded on the optical disk. A packetheader of each audio packet includes a stream ID for indicating that thestored data is a compression audio stream. When the packet stores aleading part of the audio frame, the packet header also includes an APTS(audio presentation time stamp) as audio reproduction time informationindicating the time to reproduce the frame.

FIG. 39 is a block diagram of the reproduction apparatus in the fourthexample.

Elements 501 through 532 are the same as those shown in FIG. 35 in thethird example.

Reference numeral 504 represents a buffer memory for temporarily storingthe compression audio signal. Reference numeral 541 represents audioextension means for extending the compression audio signal. Referencenumeral 542 represents a speaker for reproducing the extended audiosignal.

FIG. 40 shows a structure of the audio decoder 541. The audio decoder541 includes APTS detection means 701 for detecting the APTS stored in apacket header of the audio packet, and audio extension means 702 forextending the compression audio stream.

An operation of the optical disk reproduction apparatus shown in FIG. 39for reproducing the optical disk shown in FIG. 41 will be described.

The operation until the signal is input to the division means 505 issimilar to that with the optical disk reproduction apparatus in thethird example.

The data read from the buffer memory 504 is divided into compressionvideo signals A through C and a compression audio signal by the divisionmeans 505 and output. The division means 505 identifies which of thecompression video signals A through C and the compression audio signalis stored in each packet with the stream ID in the packet header of thepacketed data, and determines the destination based on theidentification result.

The divided compression video signals and compression audio signal aretemporarily stored in buffer memories 510 through 540 respectively.

The video decoders 511 through 531 read data from the buffer memories510 through 530 respectively, extend the compression video signals, andoutput the signals as video signals to the monitors 512 through 532respectively. The audio decoder 541 reads data from the buffer memory540, extends the compression audio signal, and outputs the signal as anaudio signal through the speaker 542.

The operations of the video decoders 511 through 531 for extending thecompression video signals and for adjusting the synchronization when thedifference between the reference time signal and the VPTS exceeds thethreshold value are the same as in the third example.

The compression audio signal read from the buffer memory 540 is input tothe audio decoder 541. The APTS detection means 701 detects and outputsthe APTS. The audio extension means 702 extends the compression audiostream and outputs the audio signal.

The APTS signal output from the audio decoder 541 is input to thereference time signal generation means 506, and the reference timesignal is adjusted by the APTS.

In the fourth example, due to the precision error of the crystaloscillator used in the reference time signal generation means 506, thevideo decoders 511 through 531 and the audio decoder 541, the referencetime signal is faster in terms of extension and reproduction relative tothe audio decoder 541. The video decoder 511 is slower and the videodecoder 521 is faster in terms of extension and reproduction relative tothe reference time signal. Unless the reproduction timing is adjusted,the reproduced video signals and audio signal are out ofsynchronization.

FIG. 42 is a timing diagram of audio reproduction in the fourth example.Part (a) of FIG. 42 shows the APTS with respect to reproduction time t.Part (b) shows the reference time signal. Part (c) shows the VPTS#A, atwhich the compression video signal A to be extended by the video decoder511 is to be reproduced, and part (d) shows the VPTS#B, at which thecompression video signal B to be extended by the video decoder 512 isreproduced.

FIG. 42 does not show the VPTS#C, at which the compression video signalC to be extended by the video decoder 531, but the diagram is almost thesame as in FIG. 38 shown regarding the third example.

The reference time signal generation means 506 is adjusted using theAPTS at time when the APTS shows ta1 and ta2, and the reference timesignal is reset as ta1 and ta2 at the respective time.

The video decoder 511 continues extension and reproduction of thecompression video signal A, and the difference between the VPTS#A andthe reference time signal exceeds 33 msec. as the threshold value at T4.Accordingly, the video reproduction timing control means of the videodecoder 511 skips one frame, which is originally to be reproduced, toadjust the reproduction timing so that the difference between the VPTS#Aand the reference time signal is equal to or less than the thresholdvalue.

The video decoder 521 continues extension and reproduction of thecompression video signal B, and the difference between the VPTS#B andthe reference time signal exceeds −33 msec. as the threshold value at T5and T6. Accordingly, the video reproduction timing control means of thevideo decoder 521 reproduces one frame in repetition, which has beenalready reproduced, to adjust the reproduction timing so that thedifference between the VPTS#B and the reference time signal is equal toor less than the threshold value.

As described above, in the fourth example, when the difference betweenthe reference time signal and the VPTS detected by each video decoderexceeds the threshold value, the video reproduction timing control meansof each video decoder performs adjustment so that difference between thereference time signal and the VPTS does not exceed the threshold value.In this manner, the video signals reproduced by video decoders can besynchronized with one another.

Regarding the difference between the reference time signal and the APTS,the APTS is not adjusted using the reference time signal but thereference time signal is adjusted using the APTS. Accordingly, audio andvideo signals are synchronized with no unnaturalness in the audiooutput.

EXAMPLE 5

The fifth example relates to a reproduction apparatus for adjusting thereference time signal using a VPTS detected by one video decoder andsynchronizing a plurality of video signals based on the reference timesignal.

FIG. 43 is a block diagram of an optical disk reproduction apparatus inthe fifth example.

Elements 501 through 532 are the same as those in the third example.Reference numeral 551 represents a video decoder used in the fifthexample.

The video decoder 551 has a function of outputting the detected VPTS.FIG. 44 shows a structure of the video decoder 551.

The video decoder 551 includes VPTS detection means 801 for detecting aVPTS indicating the reproduction time of the video signal multiplexed asthe compression video signal and video extension means 802 for extendingthe compression video signal.

In the fifth example, due to the precision error of the crystaloscillator used in the reference time signal generation means 506 andthe video decoders 521, 531 and 551, the reference time signal is fasterin terms of extension and reproduction relative to the video decoder551. The video decoder 521 is slower and the video decoder 531 is fasterin terms of extension and reproduction relative to the reference timesignal. Unless reproduction timing is adjusted, the reproduced videosignals are out of synchronization.

FIG. 45 is a timing diagram of video output in the fifth example. Part(a) of FIG. 45 shows the VPTS#A detected by the video decoder 551 withrespect to reproduction time t. Part (b) shows the reference timesignal. Part (c) shows VPTS#B, at which the compression video signal Bto be extended by the video decoder 521 is to be reproduced, and part(d) shows the VPTS#C, at which the compression video signal C to beextended by the video decoder 531 is to be reproduced.

The reference time signal generation means 506 is adjusted using theVPTS#A at time when the VPTS#A shows tv1 and tv2, and the reference timesignal is reset as tv1 and tv2 at the respective time.

The video decoder 521 continues extension and reproduction of thecompression video signal B, and the difference between the VPTS#B andthe reference time signal exceeds 33 msec. as the threshold value at T7.Accordingly, the video reproduction timing control means of the videodecoder 521 skips one frame, which is originally to be reproduced, toadjust the reproduction timing so that the difference between the VPTS#Band the reference time signal is equal to or less than the thresholdvalue.

Similarly, the video decoder 531 continues extension and reproduction ofthe compression video signal C, and the difference between the VPTS#Cand the reference time signal exceeds −33 msec. as the threshold valueat T8 and T9. Accordingly, the video reproduction timing control meansof the video decoder 531 reproduces one frame in repetition, which hasbeen already reproduced, to adjust the reproduction timing so that thedifference between the VPTS#C and the reference time signal is equal toor less than the threshold value.

As described above, in the fifth example, when the difference betweenthe reference time signal and the values of VPTSs detected by the videodecoders 521 and 531 exceeds the threshold value, the video reproductiontiming control means of each video decoder performs adjustment so thatthe difference between the reference time signal and the VPTS does notexceed the threshold value.

By adjusting the reference time signal using the VPTS#A detected by thevideo decoder 551, the video signal reproduced by the video decoder 551is not accompanied by any unnaturalness in the visual output despite theframe-by-frame skipping or repeat of the reproduction. Thus, the videosignals can be synchronized with one another.

EXAMPLE 6

The sixth example relates to a reproduction apparatus including aplurality of video decoders for extending and reproducing a compressionvideo signal. Each of the video decoders includes reference time signalgeneration means. The reproduction apparatus adjusts the reference timesignal of each video decoder using an APTS indicating the time toreproduce an audio signal to realize synchronization.

In the sixth example, the optical disk shown in FIG. 41 is used.

FIG. 46 is a block diagram of an optical disk reproduction apparatus inthe sixth example.

Elements 501 through 542 are the same as those shown in FIG. 39 in thefourth example. Unlike the reproduction apparatus shown in FIG. 39, thereproduction apparatus in this example does not have reference timesignal generation means 506 independently, but each video decoder 561through 581 has reference time signal generation means.

Reference numeral 561 represents a video decoder for extending andreproducing compression video signal A, reference numeral 571 representsa video decoder for extending and reproducing compression video signalB, and reference numeral 581 represents a video decoder for extendingand reproducing compression video signal C.

FIG. 47 shows a structure of each of the video decoders 561 through 581used in the sixth example.

The video decoder includes VPTS detection means 901 for detecting a VPTSindicating the reproduction time of the video signal multiplexed as thecompression video signal, video extension means 902 for extending thecompression video signal, and video reproduction timing control means903 for comparing the reference time signal and the VPTS and skipping orrepeating the video reproduction on a frame-by-frame basis when thecomparison result exceeds the threshold value, and reference time signalgeneration means 904 for generating the reference time signal.

In the sixth example, the reference time signal of reference time signalgeneration means 904 included in each of the video decoders 561 through581 is adjusted using the APTS detected by the audio decoder 541.

Since the reference time signals are adjusted using the same APTS, thereference time signals generated in the video decoders 561 through 581show the same value after being adjusted.

After the adjustment using the APTS, as in the fourth example, when thedifference between the reference time signal and the values of VPTSdetected by each video decoder exceeds the threshold value, the videoreproduction timing control means of each video decoder performsadjustment by skipping or repeating the reproduction on a frame-by-framebasis so that difference between the reference time signal and the VPTSdoes not exceed the threshold value.

As described above, in the sixth example, the reference time signalgenerated in each video decoder is adjusted using an APTS, and the videoreproduction timing control means of each video decoder maintains thedifference between each reference time signal and each VPTS to be equalto or less than the threshold value. Thus, the video signals can besynchronized with one another.

As in the fourth example, the audio signal and the video signal can besynchronized without providing any unnaturalness in the audio output.

In the sixth example, the reference time signals in the video decoders561 through 581 are adjusted using the APTS detected by the audiodecoder 541. The video signals can be reproduced in synchronization in asimilar manner by using one of the video decoders shown in FIG. 44 inthe fifth example and adjusting the reference time signals of the othervideo decoders using the VPTS detected by the one video decoder.

EXAMPLE 7

The seventh example relates to a reproduction apparatus forsimultaneously reproducing two compression video signals. The twocompression video signals are obtained by dividing a 3D signal into aright-eye video signal and a left-eye video signal and compressing thedivided video signals.

The overall structure of the apparatus is generally similar to that ofthe optical disk reproduction apparatus shown in FIG. 46 in the sixthexample, but the reproduction apparatus in the seventh example includestwo video decoders for extending compression video signals obtainedafter the division means 505 since two video signals are to bereproduced simultaneously. FIG. 48 shows a structure of one of the videodecoders used in the seventh example, and FIG. 49 shows a structure ofthe other video decoder used in the seventh example.

As shown in FIG. 48, the video decoder includes VPTS detection means1001 for detecting a VPTS indicating the reproduction time of the videosignal multiplexed as the compression video signal, video extensionmeans 1002 for extending the MPEG compression video signal, referencetime signal generation means 1004 for generating a reference timesignal, and video reproduction timing control means 1003 for comparingthe reference time signal and the VPTS and skipping or repeating thevideo reproduction on a frame-by-frame basis when the comparison resultexceeds the threshold value and also for outputting a horizontalsynchronization signal and a vertical synchronization signal for thevideo to be reproduced.

As shown in FIG. 49, the other video decoder includes VPTS detectionmeans 1101 for detecting a VPTS indicating the reproduction time of thevideo signal multiplexed as the compression video signal, videoextension means 1102 for extending the MPEG compression video signal,reference time signal generation means 1104 for generating a referencetime signal, and video reproduction timing control means 1103 forcomparing the reference time signal and the VPTS and skipping orrepeating the video reproduction on a frame-by-frame basis when thecomparison result exceeds the threshold value, receiving the horizontalsynchronization signal and the vertical synchronization signal for thevideo to be reproduced, and also reproducing the extended video signalin synchronization with the horizontal/vertical synchronization signals.

The video decoders are connected to each other so that the horizontalsynchronization signal and the vertical synchronization signal outputfrom the video decoder in FIG. 48 are sent to the video decoder in FIG.49.

In the optical disk reproduction apparatus in the seventh example havingthe above-described structure, the reference time signal generated byeach video decoder for the right or left eye is adjusted using an APTS,and the video reproduction timing control means of each video decodermaintains the difference between each reference time signal and eachVPTS to be equal to or less than the threshold value. Thus, theright-eye video signal and the left-eye video signal can be synchronizedwith one another on a frame-by-frame basis. By using the horizontal andvertical synchronization signals output by one of the video decoder asthe horizontal and the vertical synchronization signals of the othervideo decoder, two video signals can be reproduced in synchronization ona pixel-by-pixel basis.

In the seventh example, compression video signals obtained from a 3Dvideo signal are used and divided into the right-eye and left-eye videosignals. Alternatively, for example, an original video signal having afirst resolution is divided in a vertical and/or horizontal directioninto at least two video signals including a first video signal and asecond video signal having a second resolution which is lower than thefirst resolution. The resultant signals are compressed to be used. Thus,a plurality of video signals in synchronization with one another on apixel-by-pixel basis can be obtained as from a 3D video signal. Bysynthesizing such resultant signals, the clear original video signalhaving the original resolution is reproduced.

EXAMPLE 8

The eighth example relates to an optical disk reproduction apparatus forextending one compression video signal and two compression audio signalsand reproducing the signals simultaneously.

FIG. 52 shows a data structure of the optical disk used in the eighthexample.

Two audio signals D and E are compressed to obtain compression audiostreams D and E. A video signal is compressed to obtain a compressionvideo stream.

The compression video streams D and E and the compression video streamare packeted in units of 2 kB into audio packets and video packets. Apacket header of each packet includes a stream ID for indicating whichof the compression audio streams D and E and the compression videostream is stored, and the APTS and VPTS described above.

FIG. 50 is a block diagram of a reproduction apparatus in the eighthexample.

The reproduction apparatus has a generally similar structure to that inFIG. 39 in the fourth example. The audio decoder 541 has the samestructure as that shown in FIG. 40, and the video decoder 531 has thesame structure as that shown in FIG. 36. The audio decoder 591 has thesame structure as that shown in FIG. 51.

Reference numeral 590 represents a buffer memory for temporarily storingthe compression audio signal like the buffer memory 540. Referencenumeral 592 represents a speaker for reproducing the audio signal.

FIG. 51 shows a structure of the audio decoder 591.

The audio decoder 591 includes APTS detection means 1201 for detectingan APTS of an audio signal multiplexed as a compression audio signal,audio extension means 1202 for extending the compression audio signal,and audio reproduction timing control means 1203 for comparing thereference time signal and the APTS and skipping or repeating the audioreproduction on an audio frame-by-audio frame basis when the comparisonresult exceeds the threshold value.

A reproduction operation in the eighth example will be described.

The operation until the signal read from the optical disk 501 is inputto the division means 505 is similar to that in the other examples.

The data read from the buffer memory 504 is divided by the divisionmeans 505 into a compression video signal, the compression audio signalD and the compression audio signal E, and output. The division means 505identifies which of the compression video signal, the compression audiosignal D and the compression audio signal E is stored in each packetusing the packet ID in the packet header of the packeted data, anddetermines the destination based on the identification result.

The divided compression video signal, the compression audio signal D andcompression audio signal E are temporarily stored in buffer memories530, 540 and 590 respectively.

The video decoders reads data from the buffer memory 530, extends thecompression video signal and outputs the signal as a video signal to amonitor 532. The audio decoders 541 and 591 read data from the buffermemories 540 and 590, extend the compression audio signals and outputthe signals as audio signals through the speakers 542 and 592.

The reference time signal generated by the reference time signalgeneration means 506 is adjusted by an APTS#D detected by the audiodecoder 541.

In the audio decoder 591, an APTS#E is detected by the APTS detectionmeans 1201 and the compression audio signal E is extended by the audioextension means 1202. The audio reproduction timing control means 1203receives the extended audio signal output from the audio extension means1202, the reference time signal, and the APTS#E from the APTS detectionmeans 1201, and compares the reference time signal and the APTS#E. Whenthe difference between the reference time signal and the APTS#E exceedsthe threshold value, the audio reproduction timing control means 1203controls the audio reproduction timing so that the difference is equalto or less than the threshold value.

In the eighth example, 32 msec is used as the threshold value. The audioreproduction timing control means 1203 performs the following.

(reference time signal-APTS#E)>32 msec.:1 audio frame is skipped.

(reference time signal-APTS#E)<−32 msec.:1 audio frame is repeated.

The operation of the video decoder 531 for extending the compressionvideo signal and performing adjustment when the difference between thereference time signal and the VPTS exceeds the threshold value aresimilar to those in the third example.

In the eighth example, due to the precision error of the crystaloscillator used in the reference time signal generation means 506, thevideo decoder 531, and the audio decoders 541 and 591; the audiodecoders 541 and 591 are slower and the video decoder 531 is faster interms of extension and reproduction relative to the reference timesignal. Unless reproduction timing is adjusted, the reproduced videosignals are out of synchronization.

FIG. 53 is a timing diagram of video reproduction in the eighth example.Part (a) of FIG. 53 shows the APTS#D with respect to reproduction timet. Part (b) shows the reference time signal, part (c) shows APTS#E, atwhich the compression audio signal E to be extended by the audio decoder591 is to be reproduced, and part (d) shows the VPTS, at the compressionvideo signal to be extended by the video decoder 531 is to bereproduced. The reference time signal is adjusted using the APTS#D whenAPTS#D shows ta3 and ta4. The reference time signal is reset to ta3 andta4 at the respective time.

The audio decoder 591 continues extension and reproduction of thecompression audio signal E, and the difference between the APTS#E andthe reference time signal exceeds 32 msec. as the threshold value atT10. Accordingly, the audio reproduction timing control means 1203 ofthe audio decoder 591 skips one audio frame, which is originally to bereproduced, to adjust the reproduction timing so that the differencebetween the APTS#E and the reference time signal is equal to or lessthan the threshold value.

The difference between the VPTS and the reference time signal exceeds−33 msec. as the threshold value at T11 and T12. Accordingly, the videoreproduction timing control means of the video decoder 531 reproducesone frame in repetition, which has been already reproduced at therespective time, to adjust the reproduction timing so that thedifference between the VPTS and the reference time signal is equal to orless than the threshold value.

As described above, in the eighth example, when the difference betweenthe reference time signal and the APTS#E detected by the audio decoder591 exceeds the threshold value, the audio reproduction timing controlmeans of the audio decoder performs adjustment so that differencebetween the reference time signal and the APTS#E does not exceed thethreshold value of audio reproduction. Similarly, difference between thereference time signal and the VPTS is adjusted so as not to exceed thethreshold value of video reproduction. In this manner, each audio signaland the video signal can be synchronized with one another.

EXAMPLE 9

In the ninth example, the clock for performing extension is changed foraudio reproduction timing control.

The overall structure and operation of the reproduction apparatus in theninth example are generally similar to those of the optical diskreproduction apparatus in the eighth example, but the operation of audioreproduction timing control performed when the reference time signal andthe APTS#E exceeds the threshold value is different from that of theeighth example. With reference to FIG. 54 and 55, audio reproductiontiming control used in the ninth example will be described.

FIG. 54 shows an operation when the difference between the APTS#E andthe reference timing signal exceeds 32 msec. which is the threshold forthe audio reproduction. Part (a) of FIG. 54 shows the reference timesignal with respect to reproduction time t. Part (b) shows the APTS#E,and part (c) shows the clock frequency at which the audio decoder 591performs extension and reproduction. Ordinary extension and reproductionare performed by clock f0 having a frequency which is 384 times thesampling frequency fs of the audio signal. The difference between theAPTS#E and the reference time signal exceeds 32 msec. at time T11, andaccordingly, audio reproduction control means switches the clock f0 tof1. The frequency of clock f1 is higher by 10% than the frequency ofclock f0. Extension performed with clock f1 proceeds faster thanextension performed with clock f0 by 10%. With clock f1, the extensionis performed for 320 msec. from the point where the difference betweenthe APTS#E and the reference time signal exceeds 32 msec. which is thethreshold value. Thus, the reproduction timing is adjusted so that thedifference between the APTS#E and the reference time signal is equal toor less than the threshold value.

FIG. 55 shows an operation when the difference between the APTS#E andthe reference timing signal exceeds −32 msec. which is the threshold forthe audio reproduction. Part (a) of FIG. 55 shows the reference timesignal with respect to reproduction time t. Part (b) shows the APTS#E,and part (c) shows the clock frequency at which the audio decoder 591performs extension and reproduction.

The difference between the APTS#E and the reference time signal exceeds−32 msec. at time T12, and accordingly, audio reproduction control meansswitches the clock f0 to f2. The frequency of clock f2 is lower by 10%than the frequency of clock f0. Extension performed with clock f2proceeds more slowly than extension performed with clock f0 by 10%. Withclock f2, the extension is performed for 320 msec. from the point wherethe difference between the APTS#E and the reference time signal exceeds−32 msec. which is the threshold value. Thus, the reproduction timing isadjusted so that the difference between the APTS#E and the referencetime signal is equal to or less than the threshold value.

As described above, when the difference between the APTS#E and thereference time signal exceeds the threshold value for the audioreproduction, the clock by which the signal is extended is changed sothat the extension is performed at a higher speed or lower speed thanthe normal speed. By such an operation, the reproduction timing iscontrolled so that the difference between the APTS#E and the referencetime signal is equal to or less than the threshold value. Thus, theaudio signals and the video signal can be reproduced in synchronizationwith no unnaturalness in the audio output.

In the ninth example, the frequency of the clock is changed by 10%.Needless to say, a more natural audio signal is obtained by changing theclock less or gradually.

In the eighth and ninth examples, the reference time signal is adjustedusing the APTS#D. Alternatively, the video decoder shown in FIG. 44 canbe used, in which case the VPTS output from the video decoder can beused for adjustment.

The present invention has been described by way of specific examples.

The comparison between the reference time signal and the VPTS or APTS,control of the reproduction time, adjustment of the reproduction timingusing a VPTS or APTS can be performed by a microcomputer which controlsthe entirety of the reproduction apparatus.

In the above examples, the present invention is applied to optical diskreproduction apparatuses. The virtual frame-based editing systemaccording to the present invention is also applicable to a reproductionapparatus, referred to as the set top box, for extending compressionsignals supplied through communication networks or digital satellitebroadcasting. Even when the programs are switched, the non-continuousvideo signals are connected seamlessly, which provides a significantadvantage.

Industrial Applicability

A basic video signal and an interpolation signal are divided into frameseach having 1 GOP or more and subjected to interleaving alternately tobe recorded on the optical disk as interleave blocks. From such anoptical disk, a high resolution synthesis reproduction apparatusreproduces information in both of two types of interleave blocksalternately arranged. When the optical disk having a high resolutionvideo signal is reproduced by a non-progressive reproduction apparatus,information in the interleave block of only odd field or even field isreproduced by track jump. Thus, a complete two-dimensional video isobtained. Thus, compatibility is realized.

Especially, a high resolution video signal arrangement information fileand a high resolution picture identifier are recorded on the opticaldisk. Accordingly, the location of the high resolution video signal iseasily determined. Therefore, two ordinary interlace signals can be madeinto a progressive signal. In addition, output of different contentpictures for the right eye and left eye can be avoided.

According to the two-stream simultaneous reproduction synchronizationsystem according to the present invention, an MPEG signal can bevirtually edited on a frame-by-frame basis, which is conventionallyperformed only on a GOP-by-GOP basis if deterioration of picture qualityshould be avoided. By recording reproduction control information,signals can be connected on a frame-by-frame basis when beingreproduced. Thus, virtual frame-based editing can be realized withoutdeterioration in the picture quality.

In the two-stream simultaneous reproduction synchronization system, aplurality of compression video signals or a plurality of compressionaudio signals can be reproduced in synchronization with one anotherafter being extended.

In a reproduction apparatus in which the reference time signal isadjusted using an APTS detected by an audio decoder and the video outputtiming is controlled so that the VPTS matches the adjusted referencetime signal, an audio signal and a plurality of video signals aresynchronized for reproduction with no unnaturalness in the audio output.

In a reproduction apparatus in which the audio output timing iscontrolled by changing an extension clock, audio and video signals aresynchronized for reproduction with no unnaturalness in the audio outputwith no interruption or pause in the audio signal.

1-21. (canceled)
 22. An optical disk recording apparatus, comprising: aninput section for receiving an encoded first video stream correspondingto a first pixel number and an encoded second video stream correspondingto a second pixel number which is different from the first pixel number,wherein the first video stream includes a plurality of first interleaveunits, the second video stream includes a plurality of second interleaveunits, each of the plurality of first interleave units includes m1 GOPs(where m1 is an integer of 1 or greater), and each of the plurality ofsecond interleave units includes m2 GOPs (where m2 is an integer of 1 orgreater); a selection output section for selectively outputting theplurality of first interleave units included in the first video streamand the plurality of second interleave units included in the secondvideo stream; and a recording section for recording the signal outputfrom the selection output section on an optical disk.
 23. An opticaldisk reproduction apparatus for reproducing a signal recorded on anoptical disk, in which the optical disk has, recorded thereon, at leasta first video stream including a plurality of first GoPs and a secondvideo stream including a plurality of second GoPs, each of the pluralityof first GOPs includes a plurality of pictures, and each of theplurality of second GOPs includes a plurality of pictures, the opticaldisk reproduction apparatus comprising: a reproduction section forreproducing the first video stream and the second video stream recordedon the optical disk; a decoding section for decoding the first videostream and the second video stream; and an output section forselectively outputting the decoded first video stream and the decodedsecond video stream in accordance with reproduction control information,wherein the reproduction control information indicates that after afirst picture included in a final first GOP among the plurality of firstGOPs included in the first video stream is reproduced, a second pictureincluded in a leading second GOP among the plurality of second GOPsincluded in the second video stream is reproduced, the second picturebeing different from a leading picture of the leading second GOP.
 24. Anoptical disk reproduction apparatus according to claim 23, wherein thedecoding section starts decoding the second video stream so that thedecoding of the second picture has been completed when the reproductionof the first picture is completed.
 25. An optical disk reproductionapparatus according to claim 24, wherein: the reproduction controlinformation includes information ts1 representing a position of thefirst picture, information ts2 representing a position of the secondpicture, and information tsG representing a position of the leadingpicture of the leading second GOP, and the decoding section finds adecoding start position ta in accordance with expressionta=ts1−(ts2−tsG), and starts decoding the second video stream based onthe decoding start position ta.
 26. An optical disk reproductionapparatus according to claim 24, wherein: the reproduction controlinformation includes timing information representing the timing to startdecoding the leading second GOP so that reproduction completion time ofthe first picture matches the reproduction start time of the secondpicture, and the decoding section starts decoding the second videostream based on the timing information.
 27. An optical disk reproductionapparatus according to claim 23, wherein the decoding section omitsdecoding of a picture which is not necessary for decoding pictures fromthe leading picture of the leading second GOP to the second picture. 28.An optical disk reproduction apparatus according to claim 27, whereinthe picture which is not necessary is a B picture.
 29. An optical diskreproduction apparatus according to claim 23, further comprising abuffer memory section for storing the first video stream and the secondvideo stream, And the buffer memory section has a capacity which isequal to or greater than an amount of data of 1 GOP.
 30. An optical diskreproduction apparatus according to claim 23, wherein the reproductionsection reproduces the reproduction control information recorded on theoptical disk.
 31. An optical disk reproduction apparatus according toclaim 23, wherein: the optical disk further has, recorded thereon, anidentifier representing whether or not the reproduction controlinformation is recorded on the optical disk, and when the identifierrepresents that the reproduction control information is recorded on theoptical disk, the reproduction section reproduces the reproductioncontrol information recorded on the optical disk.
 32. An optical diskreproduction apparatus according to claim 23, wherein in a fastreproduction mode, when the second picture is not an I picture, theoutput section prohibits an I picture included in the leading second GOPfrom being output.
 33. An optical disk reproduction apparatus accordingto claim 32, wherein the output section prohibits a part of the Ipicture included in the leading second GOP from being output based on Ipicture reproduction prohibition information.
 34. A reproduction controlinformation generation apparatus, comprising: an input section forreceiving a first video stream including a plurality of first GOPs and asecond video stream including a plurality of second GOPs; and ageneration section for generating reproduction control information whichrepresents that after a first picture included in a final first GOPamong the plurality of first GOPs included in the first video stream isreproduced, a second picture included in a leading second GOP among theplurality of second GOPs included in the second video stream isreproduced, the second picture being different from a leading picture ofthe leading second GOP.
 35. A reproduction control informationgeneration apparatus according to claim 34, wherein the reproductioncontrol information includes information representing the number ofpictures from the leading picture of the leading second GOP to thesecond picture.
 36. A reproduction control information generationapparatus according to claim 34, wherein the reproduction controlinformation includes information representing the time to reproduce theleading picture of the leading second GOP and the time to reproduce thesecond picture of the leading second GOP.
 37. A reproduction controlinformation generation apparatus according to claim 34, wherein thereproduction control information includes timing informationrepresenting the timing to start decoding the leading second GOP so thatreproduction completion time of the first picture matches thereproduction start time of the second picture.
 38. A reproductioncontrol information generation apparatus according to claim 37, whereinthe timing information represents the timing to start decoding theleading second GOP when a picture which is not necessary for decodingpictures from the leading picture of the leading second GOP to thesecond picture is not decoded.
 39. A reproduction control informationgeneration apparatus according to claim 38, wherein the picture which isnot necessary is a B picture.
 40. An optical disk recording apparatus,comprising: a generation section for generating reproduction controlinformation; and a recording section for recording the reproductioncontrol information on an optical disk having, recorded thereon, a firstvideo stream including a plurality of first GOPs and a second videostream including a plurality of second GOPs, wherein the reproductioncontrol information represents that after a first picture included in afinal first GOP among the plurality of first GOPs included in the firstvideo stream is reproduced, a second picture included in a leadingsecond GOP among the plurality of second GOPs included in the secondvideo stream is reproduced, the second picture being different from aleading picture of the leading second GOP.
 41. An optical disk recordingapparatus, comprising: an editing section for editing a first videostream including a plurality of first GOPs and a second video streamincluding a plurality of second GOPs so that at least one pictureunnecessary for reproduction is deleted; and a recording section forrecording the edited first video stream and the edited second videostream on an optical disk, wherein the reproduction control informationrepresents that after a first picture included in a final first GOPamong the plurality of first GOPs included in the first video stream isreproduced, a second picture included in a leading second GOP among theplurality of second GOPs included in the second video stream isreproduced, the second picture being different from a leading picture ofthe leading second GOP.
 42. An optical recording apparatus according toclaim 41, wherein the at least one picture unnecessary for reproductionincludes a picture, of the first video stream, after the first picture,and a picture, of the second video stream, before the second picture.43. An optical recording apparatus according to claim 42, wherein the atleast one picture unnecessary for reproduction further includes apicture which is not necessary for decoding pictures from the leadingpicture of the leading second GOP in the second video stream until thesecond picture.
 44. A reproduction control information generationapparatus according to claim 43, wherein the at least one pictureunnecessary for reproduction is a B picture.
 45. An optical recordingapparatus according to claim 41, wherein the recording section recordsthe edited first video stream and the edited second video stream incontinuous regions of the optical disk.
 46. An optical recordingapparatus according to claim 41, wherein the recording section recordsthe reproduction control information on the optical disk.
 47. An opticalrecording apparatus according to claim 41, wherein the recording sectionrecords the reproduction control information on a medium other than theoptical disk.